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‘Trusting STEM Experts and Authorities in the Age of “Fake News” 


Dear SSMA Journal Readers: 

Currently, there seems to be an ongoing trend for the 
public to question the expertise and authority of STEM 
professionals (including whether they are “experts or 
authorities” at all!) by asking, “Why should we believe 
them?” As asked by Kahan, Jenkins-Smith, and Braman 
(2011, p. 1), “When empirical assessments of risk and risk 
abatement are exactly what members of the public are 
fighting about, why is the prevailing opinion of scien- 
tists—on questions only they are equipped to answer— 
so infrequently treated as decisive?” Kahan et al. (2011) 
explain that “cultural cognition,” or the idea that individ- 
uals are disposed to selectively credit or dismiss evi- 
dence of risk in patterns that fit values they share with 
others, shapes their beliefs about the existence of scien- 
tific consensus. Moreover, we are living in the so-called 
“information age,” where large amounts of information 
are widely available to many people, largely through 
computer technology. This is, in truth, also contributing 
to this phenomenon of skepticism (Alvarez, 2017). The 
question for us, as STEM educators, then, is: “How do 
we help students to become adept at circumventing the 
morass of information both good (i.e. empirical 
research) and bad (i.e. ‘Fake News’) to know what to 
believe?” 

As STEM educators, we are continuously focused on 
developing our students’ problem-solving, critical think- 
ing, and rational thinking skills. These proficiencies are 
becoming increasingly important as our world becomes 
ever more complexly interconnected. In fact, Jere H. 
Lipps (2004) argues that “Living well requires that we be 
able to evaluate our environment rationally.” Likewise, 
these are skills that STEM “experts and authorities” have 
developed and utilized to advance knowledge in the con- 
text of “cultural cognition.” 

So we must address the skepticism of people. Let us 
begin by taking a look at a few definitions: 

¢ Problem solving is the process or act of finding a 
solution to a problem (Problem solving, 2017). 
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* Critical thinking is disciplined thinking that is clear, 
rational, open-minded, and informed by evidence (Critical 
thinking, 2017). 

¢ Rational is agreeable to reason; reasonable; sensible; 
having or exercising reason, sound judgment, or good 
sense; being in or characterized by full possession of 
one’s reason; sane, lucid (Rational, 2017). 

¢ An expert is a person who has special skill or knowl- 
edge in some particular field; specialist; authority: a per- 
son possessing special skill or knowledge; trained by 
practice; skillful or skilled (Expert, 2017). 

¢ Authority. an accepted source of information 
(Authority, 2017). 

In the context of these definitions and as we continue 
to analyze with our students the questions of “What 
makes a person an expert or an authority?” as well as the 
related question of “Why should we believe them?” we 
examine criteria for “judging” experts and authority 
(Lipps, 2004): 

¢ Does the expert/authority produce independent evi- 
dence to support their claims? 

* Does the expert/authority have proper credentials 
demonstrating their expertise in the relevant field? 

¢ Does the expert/authority have proper affiliations? 

¢ Has the experts/authorities work been peer-reviewed? 

* Does the expert/authority have a past record of mak- 
ing rational claims backed by evidence? 

We look to professional organizations for guidance 
beginning with our own, the School Science and Mathe- 
matics Association (SSMA, 2011). SSMA focuses on 
building and sustaining a community of teachers, 
researchers, scientists, and mathematicians. The proper 
credentials of the members of this multinational commu- 
nity range from state teaching certification and licenses 
awarded after successfully completing rigorous, nationally 
accredited teacher education programs to tenured profes- 
sors and researchers from public and private colleges and 
universities. This membership provides the proper affilia- 
tion that warrants the authority of the members by 
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Table 1 
Alignment of Research Components 


NGSS Practices (2013) 


Asking questions (for science) and 
defining problems (for engineering) 


Developing and using models 


Planning and carrying out 
investigations 


Analyzing and interpreting data 


Using mathematics and computational 
thinking 


Constructing explanations (for 
science) and designing solutions 
(for engineering) 


Engaging in argument from evidence 


Obtaining, evaluating, and 
communicating information 


Editorial 


Standards for Quality Research (2005) 


Pose a significant, important question that 
can be investigated empirically and that 
contributes to the knowledge base. 


Base research on clear chains of inferential 
reasoning supported and justified by a 
complete coverage of the relevant 
literature. 

Ensure the study design, methods, and 
procedures are sufficiently transparent 
and ensure an independent, balanced, and 
objective approach to the research. 

Test questions that are linked to relevant 
theory. 

Apply methods that best address the 
research questions of interest. 

Use appropriate and reliable 
conceptualization and measurement of 
variables. 


Provide sufficient description of the 
sample, the intervention, and any 
comparison groups. 


Provide the necessary information to 
reproduce or replicate the study. 
Submit research to a peer-review process. 


Evaluate alternative explanations for any 
findings. 

Assess the possible impact of systematic 
bias. 


Adhere to quality standards for reporting 
(i.e., clear, cogent, complete). 


ALA Framework (2016) 


Research is iterative and depends upon ask- 
ing increasingly complex or new ques- 
tions whose answers in turn develop _ 
additional questions or lines of inquiry in 
any field. 

Searching for information is often 
nonlinear and iterative, requiring the 
evaluation of a range of information 
sources. 


Searching for information requires the 
mental flexibility to pursue alternate 
avenues as new understanding develops. 


Information possesses several 
dimensions of value, including as a 
commodity, as a means of education, as a 
means to influence, and as a means of 
negotiating and understanding the world. 
Legal and socioeconomic interests 
influence information production and 
dissemination. 

Research is iterative and depends upon ask- 
ing increasingly complex or new ques- 
tions whose answers in turn develop 
additional questions or lines of inquiry in 
any field. 

Information resources reflect their 
creators’ expertise and credibility, and 
are evaluated based on the information 
need and the context in which the 
information will be used. Authority is 
constructed in that various communities 
may recognize different types of 
authority. It is contextual in that the 
information need may help to determine 
the level of authority required. 

Communities of scholars, researchers, or 
professionals engage in sustained 
discourse with new insights and 
discoveries occurring over time as a 
result of varied perspectives and 
interpretations. 

Information in any format is produced to 
convey a message and is shared via a 
selected delivery method. The iterative 
processes of researching, creating, 
revising, and disseminating information 
vary, and the resulting product reflects 
these differences. 


eee ees 


fostering their knowledge in their field and, therefore, 


demonstrating their expertise. 


Also, SSMA works to advance knowledge through 
research in science and mathematics education and their 
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integration as well as inform practice through the dissemi- 


nation of scholarly works in and across science and math- 


ematics. Table 1 illustrates the alignment of research 
components from the Next Generation Science Standards 
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Figure 1, Interplay among research components. [Color figure can be viewed 
at wileyonlinelibrary.com] 


eight practices of science and engineering that are identi- 
fied as essential for all students to learn (NGSS, 2013); the 
Standards for Quality Research (Focus, 2005); and the 
Framework for information literacy for higher education 
(American Library Association, 2016). These practices, 
standards, and framework ensure the advancement of 
knowledge through the development of our own as well as 
our students’ problem solving, critical thinking, and 
rational thinking proficiencies. Moreover, School Science 
and Mathematics has been informing practice through the 
dissemination of scholarly manuscripts since 1901! The 
SSMA journal is an international journal emphasizing 
research. Our journal publishes articles backed by evi- 
denced determined to be rational through a rigorous peer- 
reviewed process. 

In conclusion, we can assist our students in becoming 
more adept at circumventing the morass of information 
both good and bad to know what to believe by analyzing 
with them the rigor expected of researchers and practi- 
tioners that develops “experts and authorities” to validate 
that they can be trusted. Simultaneously, we must support 
the intellectual progress of our students as they enter 
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adulthood and society by demonstrating and requiring 
from them the same level of quality documented in Table 
1. Figure 1 illustrates the interplay among these research 
components. Finally, we as STEM “experts and author- 
ities” must persist in communicating effectively with the 
public through our work in research and practice as we 
continue to influence policy in science and mathematics 
education at local, state, and national levels. 


Raymond A. Heitger 
Bowling Green State University 
Andrea R. Milner 

Adrian College 
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This study examines what students enrolled in the honors and general sections of a high school biology course offered 
at the same school learn when they have an opportunity to participate in a broad or narrow range of science practices 
during their laboratory experiences. The results of our analysis suggest that the students enrolled in the general sections 
of the course made similar or larger gains than the students enrolled in the honors section of the course in their abilities to 
plan and carry out an investigation, argue from evidence, and write a science-specific persuasive essay when these 
students had an opportunity to participate in a broad range of science practices. These findings suggest that laboratory 
experiences that give students an opportunity to participate in a broad range of science practices, although considered 
challenging by many teachers, have the potential to help all students become more proficient in science. The article 
concludes with a discussion of the implications of this study for classroom instruction and educational policy. 


The New Framework for K-12 Science Education 
(National Research Council [NRC], 2012) calls for all 
students to learn about science by “doing science.” In an 
effort to capture what “doing science” entails, the 
Framework describes a collection of practices that broadly 
represent some of the key activities that scientists engage in 
to develop new knowledge or to solve problems. These 
practices include asking questions, developing and using 
models, planning and carrying out investigations, analyzing 
and interpreting data, using mathematics, constructing 
explanations, engaging in argument from evidence, and 
obtaining, evaluating and communicating information. 
Students will need to learn how to participate in these 
important practices in order to reach the performance 
expectations found in the Next Generation Science 
Standards (NGSS) (NGSS Lead States, 2013). Teachers, as 
a result, will need to enact lessons that give students more 
opportunities to learn how to use the practices of science to 
explore and explain the natural world inside the classroom 
(Krajcik, Codere, Dahsah, Bayer, & Mun, 2014; Osborne, 
2014). However, if the current goal of science education in 
the United States is for all students to reach the NGSS 
performance expectations, this type of classroom 
instruction cannot be reserved for only a select few. This 
call for all students to learn science by doing science is 
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considered by many to be a critical and much needed shift 
in the nature of classroom instruction. The authors of the 
Framework, for example, emphasize the importance of 
making this shift in focus when they write, “arguably, the 
most pressing challenge facing U.S. education is to provide 
all students with a fair opportunity to learn” (NRC, 2012, p. 
282). 

Although there is a wide body of research that suggests 
that “all individuals, with a small number of notable 
exceptions, can engage in and learn complex subject 
matter” (NRC, 2012, p. 280), many teachers think focusing 
on complex material or using instructional approaches that 
give students an opportunity to participate in the practices 
of science is too challenging for low-performing students 
(Schmidt & McKnight, 2012). For instance, teachers often 
claim that high performing students are the only ones that 
can successfully argue from evidence because it is too 
difficult for all but the most capable students (Sampson & 
Blanchard, 2012). The instruction that teachers provide for 
students who are placed into lower track courses, as a 
result, is often fundamentally different and tends to be far 
more simplistic than the instruction that students in honors 
or advanced placement courses experience (Yerrick, 2000). 

Yerrick (2000) argues that “excellence in science 
instruction” (p. 808) is often reserved for the elite, so that 
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students in lower track sections of the same course typically 
experience instruction that focuses on the basics, limits 
students’ input and contribution during lessons, and 
exposes them to only predictable outcomes. The nature of 
science instruction that takes place in different tracks of the 
same course, as a result, often limits opportunities for 
students in the lower track course to learn and leads to 
social stratification (Gorski, 2013; Schmidt & McKnight, 
2012; Yerrick, 2000). A crucial step in closing the 
opportunity gap in our schools is to understand how the 
nature of classroom instruction can support or constrain 
opportunities for different groups of students to learn. With 
this issue in mind, the goal of this study was to explore 
what students who were tracked into different sections of a 
biology course learn when both groups of students have an 
opportunity to learn science by participating in a broad 
range of science practices. 


Theoretical Framework 

This study is grounded in social constructivist theories of 
leaming (see Anderson, 2007; Driver, Asoko, Leach, 
Mortimer, & Scott, 1994; Scott, Asoko, & Leach, 2007). This 
perspective asserts that learning science involves “people 
entering into a different way of thinking about and explaining 
the natural world; becoming socialized to a greater or lesser 
extent into the practices of the scientific community with its 
particular purposes, ways of seeing, and ways of supporting 
its knowledge claims” (Driver et al., 1994, p. 8). The learning 
of the disciplinary core ideas and the practices of science 
therefore involve both personal and social processes. The 
social process of learning involves exposure to the language, 
representations, concepts, and the practices that make science 
different from other ways of knowing. This process requires 
input and guidance about “what counts” from people that 
are familiar with the goals, norms, and epistemological 
commitments that define science as a community of practice. 
Thus, learning is dependent on supportive interactions with 
others. The individual process of learning, in contrast, 
involves the construction of new knowledge and new ways to 
explore the unknown through the appropriation of important 
ideas, modes of communication, modes of thinking, and 
practices for generating new knowledge by making sense of 
an experience and integrating new views with the old. 

This theoretical perspective has two important implications 
for instruction with respect to how students learn science 
inside the classroom. First, students must have opportunities 
to use disciplinary core ideas and participate in the practices 
of science to learn about them. Science laboratories can be an 
ideal setting for the social construction of knowledge and 
directly investigating natural phenomena using the tools, 
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models, theories, and modes of communication that are 
valued within a particular discipline (Blanchard et al., 2010). 
Second, students must develop an understanding of what 
makes certain aspects of a practice more productive or useful 
than others within a particular discipline and why. For 
example, scientists often plan to control variables and conduct 
multiple trials when designing a scientific investigation. It is 
therefore important to not only engage students in a science 
practice, such as designing an investigation, but to also help 
them understand why certain strategies or techniques 
associated with each practice, such as controlling variables or 
including multiple trials, are more productive and useful than 
others in specific situations. Without a focus on both the how 
and the why, learners never really develop a “grasp of 
practice” (Ford, 2008) that they can use in novel situations. 


Background 
Nature of Current Classroom Instruction 

Current research indicates that science teachers rarely, if 
ever, give students opportunities to participate in the 
disciplinary practices of science (Kiuhara, Graham, & 
Hweken, 2009; Weiss, Banilower, McMahon, & Smith, 
2001). This trend holds true even for laboratory 
experiences—a space in which one might expect to see 
students engaged in the practices of science. The authors 
of America’s Lab Report (NRC, 2006), for example, 
found that “the quality of current laboratory experiences is 
poor for most students” (p. 6) and that most laboratory 
experiences are highly prescriptive in nature and are 
intended to demonstrate or verify previously addressed 
content. Many science educators found these findings to 
be highly problematic at the time, especially in light of the 
calls for students to learn science through inquiry that 
were made in the Benchmarks for Science Literacy 
(American Association for the Advancement of Science 
[AAAS], 1989) and the National Science Education 
Standards (NRC, 1996) over a decade earlier. 

Recent results from the National Survey of Science and 
Mathematics Education (NSSME; Banilower et al., 2013), 
which was administered to approximately 3,700 practicing 
middle and high school science teachers in the United 
States, suggest that the nature of laboratory experiences has 
not changed much since the release of the America’s Lab 
Report in 2006. The results of the NSSME, for example, 
indicate that 80% of the teachers who completed the survey 
reported that they emphasize understanding science concepts 
in the course they teach and 50% focus on learning science 
process skills (i.e., observing and measuring), but only 18% 
report that they help their students learn to support their 
claims with evidence (p. 71, 75). In addition, the results of 
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the NSSME indicate that even though 70% of the surveyed 
teachers reported doing laboratory activities at least once a 
week, only 58% give students an opportunity to visually 
represent or analyze data, 21% encourage their students to 
reflect on learning through writing, 9% reported that they 
require their students to give oral presentations, and only 
8% engaged their students in problem-based learning 
activities (p. 76). These findings suggest that most students 
in the United States do not have the opportunity to share the 
process and products of an inquiry with others in a manner 
that is consistent with the norms and epistemological 
commitments of science. Although the practices described in 
the Framework (NRC, 2012) are considered by many 
science educators as fundamental in terms of what it means 
to do science (Indrisano & Paratore, 2005; Saul, 2004) and 
doing science is the embodiment of what some authors 
describe as authentic science learning (Crawford, 2014; 
Jiménez-Aleixandre & Erduran, 2008), few students have an 
opportunity to engage in a wide range of practices or to 
learn why various aspects of each practice are more useful or 
valuable than others when attempting to develop, test, or 
refine scientific knowledge. 
Inequitable Learning Opportunities 

While authentic laboratory experiences are uncommon in 
middle and high school science classrooms, students who 
have been tracked into general or remedial science courses 
have even fewer opportunities to learn how to participate in 
the practices of science. Yerrick (2000), for example, 
describes how high-performing students and students 
enrolled in honors or advanced sections of a course are 
expected to engage in rigorous science talk while in school, 
whereas low-performing students and students tracked into 
general or remedial sections are expected to learn a set of 
facts and basic process skills. The work of Barton and 
colleagues (2000, 2010) also draws attention to the fact that 
students in urban schools often experience instruction that 
does not give them access to the analytical and discursive 
tools of the culture of science. By enacting such instruction, 
teachers actually limit students’ ability to assume 
membership in the culture of science and their access to 
learning about and through the practices of science. This is 
problematic because Barton and Tan (2010) have shown 
that students in urban schools, who are typically labeled as 
low performing or “at-risk,” are able to quickly appropriate 
the discursive tools of science and develop a science 
identity when they have rich opportunities to participate in 
the practices of science. These findings, when taken 
together, suggest that low-performing students are 
underserved by the instruction they typically experience, 
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especially when they are tracked into the general or 
remedial sections of a science course. 

Another factor that contributes to inequitable classroom 
instruction is the teacher’s views about students (Gess- 
Newsome, Southerland, Johnston, & Woodbury, 2003). 
Sampson and Blanchard (2012), for example, found that 
teachers tend to think that the practice of arguing from 
evidence is too challenging for many students, especially 
for those students enrolled in the remedial or general 
sections of a course, so it is better to focus on the basics 
during a lesson and not structure lessons so students have 
opportunities to evaluate explanations, support claims using 
evidence, or question the reasoning of others because it 
might led to confusion. Pimentel and McNeill (2013) also 
found that the teachers in their study enacted more 
authoritarian, teacher-centered instruction when they 
thought their students did not have enough background 
knowledge or experience with argumentation even though 
they personally supported a more student-centered vision of 
instruction. Although these two studies focused mainly on 
the practice of argumentation, they demonstrate how a 
teacher’s views about different students and what type of 
instruction is most appropriate or beneficial for different 
groups of students can limit opportunities for some students 
to participate in the practices of science. 

Our review of the literature, in summary, indicates that 
the nature of current classroom instruction in science is not 
well aligned with current recommendations for making 
science more meaningful, equitable, and _ inclusive. 
Moreover, instruction for lower track students tends to be 
less cognitively demanding and leads to social stratification 
in which some students have far fewer opportunities to 
learn (Banilower et al., 2013; Schmidt & McKnight, 2012; 
Yerrick, 2000). Such unequal opportunities lead to 
questions of what students enrolled in general or remedial 
sections of a science course might learn if they had access 
to the same high quality laboratory instruction that is 
usually reserved for students enrolled in honors or 
advanced sections. 


Objective and Research Questions 

We therefore decided to examine what students who 
were enrolled in either the general or honors sections of a 
biology course learned over an academic school year when 
exposed to the same laboratory instructional approach. In 
two classrooms at one school, the teachers used a 
prescriptive approach to laboratory instruction that 
emphasized data collection and analysis. In two classrooms 
at another school, the teachers used an approach to 
laboratory instruction called Argument-Driven Inquiry, 
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Table 1 
Characteristics of the Schools 
Race (% of enrollment) Percent 2010-11 
Free or 
School Total Enrollment White Black Hispanic Asian Other Reduced Lunch School 
Eligibility Grade" 
A Loy / 60.6 30.2 3.2 322 2h 17.8 B 
B 659 50.8 31.4 9.7 2.0 6.1 28.3 A 
* School grade based on state accountability and performance outcomes for schools. 
which is designed to support students in using core ideas Description of School A 


and science practices to explain natural phenomena. The 
research questions that guided our study were: 

1. How does the biology learning of differently tracked 
students compare when laboratory experiences engage 
them in a narrow range of science practices? 

2. How does the biology learning of differently tracked 
students compare when laboratory experiences engage 
them in a broad range of science practices? 


Method 

We used a convergent mixed methods design (Creswell 
& Clark, 2011) to qualitatively describe the instruction that 
took place in two high schools and to quantitatively 
describe the student learning that occurred in each 
instructional approach for different groups of students. We 
selected these two high schools as our study sites because 
laboratory experiences were a major component of the 
biology curriculum in both schools, yet the biology teachers 
at these two schools used laboratory investigations to 
support learning in different ways. Additionally, in both 
schools, students were tracked into honors and general 
sections of the biology course. Based on these existing 
differences at each school, we wanted to describe the nature 
of laboratory instruction at each school and examine trends 
in student learning within each context. 

To describe the nature of laboratory instruction, we 
interviewed each teacher about the number of laboratory 
experiences in their curriculum, when these experiences 
were provided to their students, and what they expected 
their students to do during each one. We also collected 
copies of their lesson plans and instructional materials from 
each teacher as part of the interview. Finally, we conducted 
observations of classroom practice over the course of the 
school year. To examine student learning, we administered 
three assessments at the beginning and end of the school 
year: a content assessment, a performance task, and a 
writing assessment. 


School Science and Mathematics 


School A was a 9-12 public high school in the 
southeastern United States. Table 1 displays the breakdown 
of enrollment, race, free or reduced lunch eligibility, and 
school grade. Mike and Betty taught Biology at School A. 
Mike had been teaching at school A for one year before the 
year of this study, but he had 25 years of experience 
teaching, having taught high school biology, chemistry, and 
physics. During this study, he taught four sections of 
general biology. Betty had worked at this school for several 
years at the time of this study, with a high school teaching 
career lasting 11 years. During this study, she taught five 
sections of biology, two of them being honors sections. All 
students enrolled in the biology course during the 2011—12 
academic year were asked to participate in the study and 
a total of 100 students agreed. However, owing to high 
school attrition during the year and high student 
absenteeism on one of the two days allowed by the 
assessment administration, complete datasets were 
collected from 31 students (15 general; 16 honors) for the 
biology content assessment, 30 students (15 general; 15 
honors) for the performance task assessment, and 49 
students (22 general; 27 honors) for the biology writing 
assessment. These students were either in 9th or 10th grade. 
Description of School B 

School B was a 9-12 charter school in the southeastern 
United States. Table 1 displays the breakdown of 
enrollment, race, free or reduced lunch eligibility, and 
school grade. Helen and Willa taught biology at School B. 
Helen had been at this school for 15 years, teaching general 
biology, honors biology, and anatomy courses. During this 
study, she taught two sections of honors biology. Willa had 
seven years of teaching experience, one year at this school, 
teaching general and honors biology. During this study, she 
taught five sections of biology, including one honors 
section. All students enrolled in the biology course during 
the 2011-12 academic year were asked to participate in the 
study and 117 students agreed; however, owing to 
incomplete forms and student absenteeism, complete 
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datasets were collected from 99 students (54 honors; 45 
general) for the biology content assessment, 89 students (51 
honors, 38 general) for the performance task assessment, 
and 103 students (57 honors; 46 general) for the biology 
writing assessment. These students were either in 9th or 
10th grade. 

Instruments 

Student learning over the academic year was measured 
using three assessments that were each administered at the 
outset and end of the school year. These assessments were 
developed in conjunction with tenured biology faculty at a 
nearby research university to ensure content and construct 
validity. The biologists also provided an “expert answer” to 
each question that we used to construct the itemized rubrics 
and ensure translational validity. See Strimaitis, Enderle, 
Grooms, and Sampson (2015) for the development and 
validation process for each instrument. 

Content assessment. The content instrument assessed 
student knowledge and application of biology concepts. 
The pre-test and the post-test were identical, consisting of 
eleven free response questions. Each question related to a 
core idea identified in state and national standards 
documents for high school biology that was addressed in 
the curriculum at each school. The core ideas were cell 
theory, mitosis, photosynthesis, cellular respiration, patterns 
of inheritance, meiosis, DNA structure, macroevolution, 
biological species concept, mechanisms of evolution, and 
nutrient cycles. Each question provided background 
information related to the core idea, followed by two 
questions. The first question asked the student to describe 
the science concept underlying the question. The second 
question asked the student to apply understanding of that 
science concept. The response to each question was scored 
as either 0 points (inaccurate response), 1 point (insufficient 
but accurate), 2 points (adequate and accurate), or 3 points 
(nuanced response) based on the accuracy and the number 
of elements from the expert answer included in the student 
answer. Scores on instrument ranged from 0 to 72 points. 

Performance task. This instrument prompted students 
to plan and carry out an investigation, analyze and interpret 
data, develop an explanation, and argue from evidence. The 
task was completed individually and included a short 
paragraph describing a science concept, a_ research 
question, and a list of materials available to the student. 
Open-ended questions prompted the student to record and 
explain his or her decision-making processes and the results 
of those decisions. The performance task was scored out of 
a maximum of 32 points and included subscales to 
separately assess each student’s proficiency in each of the 
four practices. The pre-test was based on the guiding 
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question: What kind of animal did the unknown specimen 
come from: a predatory bird, a perching bird, or something 
else? The available materials included footprints from an 
unknown animal, predatory birds, and perching birds, and 
various data collection materials. The post-test was based 
on the guiding question: Did the unknown skull come from 
a carnivore, herbivore, or something else? The available 
materials included skulls from an unknown animal, from 
herbivores, and from carnivores, and various data collection 
materials. 

Writing assessment. The writing assessment tasked 
students to argue from evidence, evaluate information, and 
communicate information through writing. The assessment 
included background information, a dataset, and an 
inaccurate argument generated from the dataset. The prompt 
directed each student to present a countering argument in 
response to the inaccurate argument, including appropriate 
evidence and justification. Photosynthesis was the topic of 
both the pre-test and post-test. The assessment was scored 
out of a maximum of 26 points and included subscales to 
assess the structure of the written argument (0—6 points), the 
content of the argument (0-10 points), and the writing 
mechanics (0-10 points). 

Data Analysis 

We collected information during the teacher interviews to 
help us understand the nature of laboratory instruction at 
each school. Our first step was to determine the number of 
laboratory experiences that each teacher provided and if the 
number differed by section of the course. Next, we used the 
framework from Blanchard et al. (2010) to describe the 
nature of classroom inquiry to classify each laboratory 
experience. We then developed codes to identify how many 
different opportunities the students had to participate in 
the science engineering practices (SEP) found in the 
Framework (NRC, 2012) during each laboratory 
experience. We focused on the practices of planning and 
carrying out investigations (SEP #3), analyzing and 
interpreting data (SEP #4), constructing explanation (SEP 
#6), and obtaining, evaluating, and communicating 
information (SEP #8) for this analysis due to the nature of 
the laboratory experiences. Since many of these practices 
are complex and consist of multiple components, we used 
several different codes to identify the various components 
of these practices. For example, we used three different 
codes to identify different activities related to the practice of 
obtaining, evaluating, and communicating information. 
These included (a) evaluating information presented to 
them, (b) communicating information through an oral 
presentation, and (c) communicating information through a 
written report. Finally, we used the classroom observations 
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Type of Laboratory Experiences (Inquiry Level)* 


Table 2 
The Number and Type of Laboratory Experiences Provided in Each Instructional Approach 
Approach Teacher Class Sections Number 
of Lab 
General Honors Experiences 

Narrow Mike 4 0 30 

Betty 5 2 15 
Broad Helen 0 2 12 

Willa 4 1 9 


Activity (0) Structured (1) Guided (2) Open (3) 
19 (63%) 8 (27%) 2 (7%) 1 (3%) 
13 (87%) 1 (7%) 1 (7%) 0 (0%) 

0 (0%) 0 (0%) 12 (100%) 0 (0%) 
0 (0%) 0 (0%) 9 (100%) 0 (0%) 


* Inquiry levels for laboratory experiences described in Blanchard et al. (2010). 


that we conducted during the academic year to confirm that 
our analysis of the teachers’ comments during the 
interviews, lesson plans, and instructional materials were 
consistent with what actually happened inside the classroom. 

A triad of researchers scored 25% of the fully blinded set 
of each assessment and the intraclass correlation coefficient 
(ICC) was determined (two-way random effects, absolute 
agreement). An ICC above .6 is considered substantial 
agreement (Landis & Koch, 1977), and once obtained, the 
team members scored individually. The ICC for the 
content, performance task, and writing assessments were 
.790, .853, and .794, respectively. Paired-samples ¢-tests 
were used to quantify gains on each assessment. Cohen’s d 
effect sizes were calculated and interpreted small (.2), 
moderate (.5), and large (.8) effects, respectively (Cohen, 
1992). 


Results 
Nature of Instruction in Each School 

Mike and Betty reported that their biology students 
participated in a high number of laboratory experiences 
(15-30) during the year. Most of these laboratory 
experiences were verification activities, or Level 0 inquiry 
(Table 2). As shown in Table 3, the majority of these 
activities involved data analysis and interpretation, but few 
involved designing methods to collect data, arguing from 
evidence or giving an oral presentation. Because students 
had an opportunity to participate in a narrow range of 
science practices, this instructional approach is referred to 
as “narrow” in all tables and figures. It is important to note 
that Betty used the same laboratory experiences in both her 
general and honors sections of the course. 

Helen and Willa, in contrast, planned laboratory 
experiences using an instructional model called Argument- 
Driven Inquiry (ADI) (Sampson et al., 2014). Helen and 
Willa described all their biology laboratory experiences as 
guided or level 2 inquiry investigations (Table 2). 
Blanchard et al. (2010) describes level 2 inquiry as an 
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investigation where the teacher provides the investigation 
question, but the data collection methods and interpretation 
of results are open to the student. Each laboratory 
investigation spanned several class periods and students 
engaged in several practices of science. As shown in Table 
3, every investigation involved students in planning a 
method to collect data, analyzing and interpreting data, 
arguing from evidence, evaluating information, and giving 
an oral presentation. Additionally, almost every 
investigation required the students to write about their 
investigation and its outcomes. Because students had an 
opportunity to participate in a broad range of science 
practices, this instructional approach is referred to as 
“broad” in all tables and figures. Willa, much like Betty, 
used the same laboratory experiences in both her general 
and honors sections of the course. 

Student Learning Gains 

We examined student gains on each assessment by 
grouping students according to enrollment in honors and 
general class sections. Table 4 displays the results of the 
paired-samples t-tests for the student groups for each 
instructional approach on each assessment. Figure 1 
displays the mean pre-test and post-test scores on the (a) 
content knowledge assessment, (b) performance task, and 
(c) writing assessment for students enrolled in the honors 
and general biology sections. 

Students enrolled in both the honors and general biology 
classes at school A (narrow instructional approach) made 
significant gains on the content knowledge assessment. 
This instrument, as noted earlier, measured their knowledge 
of and ability to use the big ideas in biology (see Table 4 
and Figure la). The effect size for each group was large 
(Cohen, 1992). Students enrolled in the honors classes 
made significant but moderate gains on the performance 
task assessment. The students enrolled in the general 
classes, however, did not improve in their ability to plan 
and carry out a scientific investigation, analyze and 
interpret data, and construct a scientific explanation (see 
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Table 3 


Opportunities to Participate in the Practices of Science within each Instructional Approach 


Opportunities to Participate in Scientific Practices (NGSS Practice #)* 


Communicate 
Information 


(written) 
(SEP # 8) 


Information 
(oral) 
(SEP #8) 


Communicate 


Evaluate 
Information 
(SEP #8) 


Evidence 
(SEP #7) 


Argue from 


Construct 
Explanations 
(SEP #6) 


Analyze and 
Interpret Data 
(SEP #4) 


Plan an 
investigation 
(SEP #3) 


Number of 
Lab 
Experiences 


Teacher 


12 (40%) 
1 (7%) 
10 (83%) 
7 (78%) 


3 (10%) 
1 (7%) 
12 (100%) 
9 (100%) 


3 (10%) 
0 (0%) 
12 (100%) 
9 (100%) 


6 (20%) 
1 (7%) 


12 (100%) 
9 (100%) 


11 (37%) 
1 (7%) 
12 (100%) 
9 (100%) 


a 
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* We did not ask teachers about practice 1, 2, and 5 in this study. 
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Table 4 and Figure 1b). Neither the honors group nor the 
general group made a significant gain on the biology 
writing assessment (see Table 4 and Figure Ic). 

Students enrolled in the honors and general sections of 
the biology course offered at school B (broad instructional 
approach) made significant and large gains on the content 
knowledge assessment (see Table 4 and Figure 1a) much 
like the students at school A did. The students enrolled in 
the honors and general sections of biology at school B, 
however, also made significant and moderate gains on the 
performance task assessment (see Table 4 and Figure 1b). 
In addition, both groups of students at school B made 
significant but small gains on the writing assessment (see 
Table 4 and Figure Ic). 


Limitations and Discussion 

We focused on student growth in both content 
knowledge and in a broad range of science practices, as 
described in the Framework (NRC, 2012). It was important 
to focus on multiple aspects of student learning because 
different instructional approaches may promote or constrain 
different aspects of student learning and few studies have 
examined differences in learning for different groups of 
students using multiple measures. This choice, however, 
required us to develop and administer three assessment 
instruments at the beginning and end of the school year 
because existing measures used by the school and the state 
only focus on content knowledge. Giving these assessments 
required investment of potential instructional time. The 
teachers using the narrow instructional approach allocated 
two days to administer the three assessments. We therefore 
could only administer two of the three assessments to each 
student. This constraint, in addition to a high rate of 
absenteeism on the testing days, resulted in a small student 
sample for the narrow instructional approach. With these 
limitations in mind, we will now provide a tentative answer 
for each research question based on the results that we 
provided earlier. 

How Does the Biology Learning of Differently 
Tracked Students Compare When Laboratory 
Experiences Engage Them in a Narrow Range of 
Science Practices? 

Mike and Betty described the nature of the laboratory 
experiences they provided to their students as largely 
confirmation labs that could be described as Level 0 or 
Level 1 inquiry (see Table 1). Students regularly engaged in 
analyzing and interpreting data during laboratory 
instruction, but they had limited opportunities to participate 
in the other science practices (see Table 2). Mike and 
Betty’s qualitative descriptions reflect national trends 
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Table 4 
Results of Paired-samples t-test Analyses for Honors and General Classes for Each Instructional Approach on Each Assessment 
Instructional Assessment Class Section Pre Mean SD Post Mean SD t statistic yy) d 
Approach 
Narrow Content Knowledge — Honors 4.22 3.51 O11 4.81 1(17) = 5.52 <.001* 1.30 
General 85 .90 3.85 313 (12) = 3.95 .002* 1.10 
Performance Task Honors V2) 2A) 14.80 6.06 #14)=2.92 LaF io 
General 10.73 3.49 10.93 494 4#14)=.19 856 _ 
Writing Assessment Honors 13.44 sz 13.96 4.51 (26) = .52 .605 - 
General 11.59 5.82 12.50 Std (21) = .60 554 - 
Broad Content Knowledge —_ Honors 4.65 3.10 19.07». 5.79. .K53)= 18,1 Ze OL ut 40 
General 2.91 2.03 13.62 837 1#44)=9.69 <.001* 1.45 
Performance Task Honors TSt2 4.70 2B Sesh 25 (50) = 4.40 <.001* .62 
General 12.38 5.68 15.78 4.20 136)=3.30 .002* 54 
Writing Assessment — Honors 16.23 4.18 L729 3.91 (56) = 2.24 .029* 30 
General 11.35 4.26 13.24 3.84 4#51)=6.12 O12" 39 





*Significant at the p < .05 level. 


+Cohen’s d values of .2, .5, and .8 are interpreted as small, medium, and large effect sizes, respectively. 


describing the nature of instruction in science classrooms 
(Banilower et al., 2013). When students participated in such 
laboratory experiences, students grouped in honors and 
general sections made significant large gains in biology 
content knowledge (see Figure la). Given the focus on 
confirming biology content through laboratory experiences, 
the results that both groups of students learned biology 
content when the instructional approach focused on 
learning content are not surprising. Content knowledge is 
generally the aspect of science proficiency that is privileged 
on tests developed at the school, state, and national levels 
(Corliss & Linn, 2011). 

We did not expect students who participated in 
laboratory experiences that focused on a narrow range of 
science practices to make large gains on the performance 
task or the writing assessment because they did not have 
much opportunity to participate in the assessed practices 
(see Table 2). A large bedy of literature has established that 
most students do not learn how to plan and carry out an 
investigation, construct a scientific explanation, argue from 
evidence, or write about science content in a manner that is 
consistent with norms and epistemological commitments of 
the scientific community if they do have an opportunity to 
engage in these practices during science instruction (Barton 
& Tan, 2010; Crawford, 2014; Wilson, Taylor, Kowalski, 
& Carlson, 2010). In this study, the students in the honors 
sections and the general sections did not make significant 
gains on the biology writing assessment (see Figure Ic). 
However, the students enrolled in the honors section did 
make a significant gain on the performance task assessment 
with a moderate effect size (see Figure 1b). 
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One potential explanation for these findings is that the 
narrow instructional approach is meeting the needs of 
students in the honors class, but not the students in the 
general class. The students tracked into the honors class 
appear to also be the students who perform well on content 
tests, which assess how a student can recall and apply 
biology concepts. The students in the honors classes may 
have been able to recall learning about how to design and 
carry out an experiment to construct a scientific explanation 
from previous readings or lectures and then applied this 
information on the performance task, without regularly 
engaging in these science practices during classroom 
instruction. Perhaps the students who are tracked into the 
general classes needed to engage in science practices to 
learn how to participate in them. 

It was important to document what students learned in this 
context because these data provide a baseline of what students 
are likely to lear in a classroom where laboratory experiences 
are consistent with the national trends in instruction (Banilower 
et al., 2013). This type of instruction, as noted earlier, is not 
aligned with the vision of instruction outlined in the 
Framework (NRC, 2012). Such instruction appears to serve all 
students in developing biology content knowledge, but it only 
serves students that already have strong content knowledge in 
developing proficiency at science practices such as planning 
and carrying out investigations, analyzing and interpreting data, 
constructing explanations, and arguing from evidence. This 
type of instruction, as a result, may actually contribute to the 
performance gap between students tracked into the honors 
biology classes and students tracked into the general biology 
sections. 
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Figure I, Pre-test and post-test scores for student groups based on class 
enrollment on the (a) content knowledge assessment, (b) performance task 
assessment, and (c) writing assessment. Error bars represent +1 SE; 
d= effect size for paired-samples t-test; n = sample size. 


How Does the Biology Learning of Differently 
Tracked Students Compare When Laboratory 
Experiences Engage Them in a Broad Range of 
Science Practices? 

Helen and Willa described the nature of the laboratory 
experiences they provided to their students as entirely 
guided (level 2) inquiry (Table 1). These students also had 
an opportunity to participate in a broad range of science 
practices during each one of their laboratory experiences 
(Table 2). The students in this context, in other words, had 
multiple opportunities to use core ideas to design a method 
to collect data, analyze and interpret data, construct an 
explanation, argue from evidence, and communicate about 
their decision-making processes and the results of those 
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processes) in an attempt to explore and explain the natural 
world. Helen and Willa’s qualitative descriptions of their 
laboratory instructional practices, as a result, were well 
aligned with the vision of instruction outlined in the 
Framework (NRC, 2012). 

When students engaged in a broad range of science 
practices to investigate core ideas, both the honors and 
general class sections made significant large gains in 
biology content knowledge (see Figure 1a). In addition, the 
gains made by students enrolled in both the honors and 
general sections of the course on the content knowledge 
assessment were much greater, in terms of effect sizes, than 
the gains made by the students in the honors and general 
sections who only had an opportunity to participate in a 
natrow range of practices. This is notable because it 
indicates that all students can improve their content 
knowledge when they have opportunities to use core ideas 
and the practices of science to explore and explain natural 
phenomena. Moreover, these results challenge the wide- 
spread concern that incorporating a greater focus on science 
practices into the science curriculum, as recommended by 
the Framework (NRC, 2012), will take away from the time 
needed to teach mandated science content (Jones & 
Johnston, 2002). Our study indicates that engaging students 
in a broad range of science practice to investigate core 
disciplinary ideas supports biology content learning for all 
student groups. 

Additionally, both the students enrolled in the honors 
sections and the general sections of the course made similar 
gains on both the performance task and the writing prompt 
(see Figures 1b and 1c) when they had an opportunity to 
participate in laboratory experiences that engaged them in a 
broad range of scientific practices. While the students 
enrolled in honors and general biology sections started at 
different places, both groups exhibited the same patterns of 
growth. In addition, the students enrolled in general 
sections of biology at school B (broad _ instructional 
approach) scored at the same level as the students enrolled 
in the honors sections of biology at school A (narrow 
instructional approach) on both the performance task and 
writing prompt at the end of the year (see Figures 1b and 
Ic). Although these trends will require additional studies 
using a different research design to substantiate, they 
suggest that engaging students in a broad range of scientific 
practices during laboratory experience has the potential to 
erase achievement gaps and make biology instruction more 
equitable. These descriptions of what different groups of 
students learned when engaged in a broad range of science 
practices also illustrate how all students, regardless of what 
course they are tracked into, are capable of engaging in and 
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developing proficiency in the practices of science while 
also learning content knowledge. 
Implications for Teaching and Learning Science 

Our findings address the widespread concern that 
engaging students in practices of science takes away from 
the instructional time that is needed to help student learn the 
content that is a focus of mandated state science 
assessments. Teachers and administrators often worry that 
using instructional time to engage students in science 
practices will lead to lower scores on these assessments 
because these assessment are designed to measure students’ 
knowledge of concepts, facts, and vocabulary (Jones & 
Johnston, 2002). Yet, our findings call into question such a 
view. Our findings suggest that when students engage in a 
wide range of scientific practices as part of an authentic 
investigation, they are able to learn science content at the 
same time, as evidenced in the broad instructional 
approach. This finding is consistent with other studies that 
indicate that learning content through inquiry-based 
instructional approaches fosters content learning for both 
high and low performing students rather than hindering it 
(Blanchard et al., 2010; Wilson et al., 2010). 

These findings are also important because the 
performance expectations found in the NGSS (NGSS Lead 
States, 2013), which are based on the Framework (2012), 
require students to be able to use core ideas and the 
practices of science at the same time. This shift in focus will 
require teachers to focus on both content and practices 
inside the classroom. Although students can learn a great 
deal of content when teachers use a more traditional 
approach to laboratory instruction, students will not learn 
how to participate in the practices of science unless these 
practices become a focus of classroom instruction. These 
findings suggest that instructional time can be used more 
efficiently when teachers focus on the core ideas and 
practices of science at the same time even though these 
types of lessons may span multiple class periods. 

Furthermore, the significant learning gains displayed by 
the general biology class sections in the broad instructional 
approach on all three assessments contradict the long-held 
concern among many teachers that lessons that engage 
students in the practices of science are too challenging for 
some students (Barton, 2003; Pimentel & McNeill, 2013; 
Prime & Miranda, 2006; Sampson & Blanchard, 2012). 
These findings add quantitative support to qualitative 
studies that demonstrated how engaging in science 
practices can improve student understanding of and 
proficiency with them, while also developing identities and 
communities that privilege the epistemic criteria underlying 
them (Barton & Tan, 2010; Ryu & Sandoval, 2012). Our 
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findings are also important because they can be used to 
challenge teachers’ views about different students and what 
type of instruction is most appropriate or beneficial for 
different groups of students. 

These findings are also important because many past 
attempts to close achievement gaps have failed to be 
successful in terms of addressing the needs of all students. 
Rather than closing achievement gaps, many approaches 
such as tracking or focusing on the basics have created 
opportunity gaps that have placed many groups of students 
at a greater disadvantage (Lee & Buxton, 2010). As we 
work to enact the vision of science education described in 
the Framework (NRC, 2012), all students must be given the 
same opportunity to learn science by engaging in the 
practices of science, and teachers will need to make sure all 
students have access to this type of instruction. One way to 
accomplish this goal is to encourage science teachers to use 
new instructional approaches, such as Argument-Driven 
Inquiry (Sampson et al., 2014), that are specifically 
designed to give students an opportunity to learn how to 
use core ideas and the practices of science to explore and 
explain the natural world. These types of instructional 
approaches not only provide teachers with a tool they can 
use to design lessons that are well aligned with the vision of 
science instruction found in the Framework (NRC, 2012), 
but they also provide teachers with a way to make their 
lessons more equitable for the diverse group of students in 
their classrooms. 

Implication for Science Education Policy 

It is common practice nationwide to group students by 
perceived ability or past performance on standardized 
assessments. This practice is often described as tracking. It 
is intended to be beneficial for both students and teachers 
because instruction can be tailored to the needs of students 
at different levels and it is easier for teachers to create a 
lesson for a group of students who are all at the same level. 
For example, teachers can provide extra time or different 
lessons for students in one class who are struggling and 
provide extra challenges for the students in another who 
mastered the content quickly. However, there is little 
evidence that tracking benefits either low- or high- 
performing students (Carbonaro, 2005). Instead, current 
research suggests that the instruction for “lower track” 
students tends to be less cognitively demanding, gives 
students fewer opportunities to learn, and leads to social 
stratification (Banilower et al., 2013; Schmidt & McKnight, 
2012; Yerrick, 2000) 

Our findings from the broad instructional approach 
further call into question the assumed benefits of tracking. 
When students enrolled in the honors and general sections 
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of the same course experienced the same laboratory 
instructional approach, both groups made similar learning 
gains. It appears that there is little reason to sort students 
into honors and general class sections when all students are 
capable of engaging in and learning from rigorous, 
cognitively demanding biological instruction. In addition, 
the two teachers in our study who taught both honors and 
general sections of biology (Betty and Willa) reported that 
they used the same laboratory experiences in both sections. 
Therefore, these two teachers did not make substantial 
modifications to their lessons in order to better meet the 
needs of their students enrolled in the different sections of 
the course (which is an assumed benefit of tracking). Betty 
and Willa’s decision to use the same laboratory experiences 
in both sections of the course, however, lead to very 
different outcomes. In the case of Betty, who used an 
instructional approach that engaged students in a narrow 
range of practices, the students enrolled in the general 
sections of the course learned much less than their peers in 
the honor section. Willa, in contrast, used an instructional 
approach that engaged students in a broad range of 
practices in both sections. This decision enabled the 
students in the general section to make the same gains as 
the students enrolled in the honors section did. This finding 
is important because it suggests that a teacher’s decision to 
use the same type of instruction for all students can either 
make the classroom more equitable or it can place some 
students at a disadvantage depending on the nature of the 
instructional approach the teacher decides to use. 
Implications for Future Research 

As current efforts call for science curricula and 
instruction to emphasize the coordination of science content 
and practices, teachers must make instructional decisions to 
accomplish this task (Krajcik et al., 2014). Undoubtedly, 
establishing classroom communities that reflect the norms 
and epistemological commitments of science as a discipline 
will require teachers to adopt new approaches to instruction 
(Bybee, 2014; NRC, 2012). Teachers, as a result, must 
make choices about what to emphasize inside the classroom 
and what type of instructional materials or approaches to 
use to support student learning, and these decisions will 
have a major influence on what students do inside the 
classroom (Barton, 2003; Pimentel & McNeill, 2013; Prime 
& Miranda, 2006). In this study, Helen and Willa enacted 
an instructional model that featured broad engagement in 
scientific practices. It is likely, however, that many teachers 
may face challenges with such instruction and may lack the 
background experiences to successfully emphasize science 
practices in their classroom (Pruitt, 2014). It is clear that 
teachers play a crucial role in this type of instruction 
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(Bybee, 2014); therefore, future studies should investigate 
the specific teacher actions and scaffolds used by teachers 
when they engage their students in science practices. 
Additionally, understanding the background experiences 
teachers bring to the teaching context, such as their 
pedagogical content knowledge related to incorporating 
specific science practices (McNeill, Gonzalez-Howard, 
Katsh-Singer, & Loper, 2015), will inform the nature of 
support that teachers need to establish classroom 
environments that promote student learning through 
engagement in scientific practices. 
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A national study was conducted to investigate the inclusion of environmental education (EE) into educator preparation 
programs across the United States. Responses from teacher educators in institutions of higher education indicated that the 
infusion of EE into educator preparation programs is limited and varies greatly across the nation. Findings also identified 
more barriers than supports for the inclusion of EE, and there exists minimal knowledge and use of the North American 
Association for Environmental Education Guidelines for the Preparation and Professional Development of Environmental 
Educators by teacher educators for programmatic development and assessment. The results also indicated that most state 
licensure programs have few or no requirements related to environmental education, and in the majority of educator 


preparation programs, environmental education is not institutionalized. 


A goal of environmental education (EE) is to develop a 
world citizenry that has the “knowledge, skills, attitudes, 
motivations, and commitment to work individually and 
collectively toward solutions of current problems and the 
prevention of new ones” (UNESCO-UNEP, 1976, p. 3). 
Through the years, this goal has led to an increased 
emphasis on the inclusion of EE in formal education and, 
consequently, an increasing need for effective EE 
teachers (McDonald & Dominguez, 2010; Moseley, 
Desjean-Perrotta, & Crim, 2014). Emphasis on the 
inclusion of EE curricula and teaching strategies in 
educator preparation programs (EPPs) is _ essential 
(Cutter-Mackenzie & Smith, 2003; McKeown-Ice, 2000; 
Tuncer et al., 2009). 

The importance of EE in teacher education has been well 
established globally for over four decades. International 
support for including EE in teacher education can be traced 
back to the Belgrade Charter (UNESCO-UNEP, 1976) and 
the Tbilisi Declaration (UNESCO-UNEP, 1978). Almost 
three decades ago, the United Nations Educational, 
Scientific, and Cultural Organization (UNESCO) described 
the preparation of environmentally literate teachers as a top 
priority (UNESCO-UNEP, 1990). In 2002, the United 
Nations General Assembly declared the time period 
between 2005 and 2014 as the Decade of Education for 
Sustainable Development (DESD) in order to emphasize 
the critical role of education and teachers in moving toward 
a more sustainable world. In addition, the UNESCO World 
Conference on Education for Sustainable Development 
(2009) called for efforts to integrate EE into both preservice 
and inservice programs. 
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Rationale for the Study 

Research shows that classroom teachers are a strong 
influence on how children perceive their role in the 
environment (Rickinson, 2001; Shepardson, Wee, Priddy, 
& Harbor, 2005) and play a critical role in advancing 
environmental literacy in schools (Alvarez-Garcia, Sureda- 
Negre, & Comas-Forgas, 2015; Hestness, McGinnis, 
Riedinger, & Marbach-Ad, 2011; Karpudewan & Ismail, 
2012). However, previous studies indicate that preservice 
teachers often have limited environmental knowledge (Puk 
& Stibbards, 2010; Tuncer, Boone, Tuzun & Oztekin, 
2014) and hold misconceptions about the environment and 
environmental issues (Khalid, 2003). For example, in several 
prior studies, preservice teachers held the misconceptions 
that holes in the ozone layer are a direct cause of global 
warming (Summers, Kruger, Childs, & Mant, 2000), ozone 
layer depletion is directly related to the greenhouse effect 
(Michail, Stamou, & Stamou, 2007), and pollutant gases are 
responsible for creating the greenhouse effect (Boon, 2010). 
These misconceptions may exist because of the complex and 
abstract nature of the concepts, the misrepresentation of these 
issues in the media, or inaccuracies in textbooks (McDonald 
& Dominguez, 2010). 

Additionally, research in science education indicates that 
teachers who hold negative attitudes toward science have a 
greater tendency to pass these attitudes on to their students 
(Osborne, Simon, & Collins, 2003). We believe the same 
potential exists for teachers in relaying negative attitudes 
about the environment to their students. There exists a 
growing body of literature that indicates pro-environmental 
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behavior is a function of environmental literacy and may be 
learned through increased environmental knowledge (Peer, 
Goldman, & Yavetz, 2007). Likewise, previous studies 
indicate a correlation of environmental knowledge to 
positive environmental attitude (Tikka, Kuitunen, & Tynys, 
2000) and an association between attitude and behavior 
(Hsu, 2004). Individuals may change their environmental 
behavior when their attitudes and beliefs change (Dietz, 
Fitzgerald, & Shwom, 2005), potentially resulting in 
increased environmental literacy. 

Teachers’ attitudes strongly influence their classroom 
practice and content (Sund & Wickman, 2008) as they are 
more likely to teach material in which they are interested 
and knowledgeable (Haney & McArthur, 2002; Roehrig & 
Kruse, 2005). Because the classroom teacher is one of the 
primary environmental educators for children, it is critical 
that EPPs prepare preservice teachers to effectively teach EE 
(Rickinson, 2001). Champeau (1990) found in his study that 
preservice teachers who receive training in EE during their 
EPPs are more likely to incorporate EE into their teaching. 
However, a meta-analysis of research studies in EE 
concluded that preservice teachers, despite having positive 
attitudes toward EE, acquire limited environmental 
knowledge during their higher education training (Alvarez- 
Garcia et al., 2015). Teacher educators can assist preservice 
teachers in understanding how their attitudes toward and 
knowledge about the environment influence their future 
teaching, exponentially impacting the environmental literacy 
of the children in their future classrooms (Powers, 2004). 

Insufficient teacher preparation in EE has been identified in 
previous studies as a limitation in past EE efforts (Alvarez- 
Garcia et al., 2015; Quinn, Littledyke, Taylor, & Davis, 
2010). However, previous studies identified several 
challenges that need to be addressed by EEPs related to EE in 
teacher preparation programs. These included lack of time for 
preparation and implementation (Paige, Lloyd, & Chartres, 
2008), multiple priorities competing for time and attention 
(Moore, 2005), lack of communication among subject areas 
(Littledyke, Taylor, & Eames, 2009), lack of content, 
pedagogical knowledge, and skills in teaching in outdoor 
settings (Kim & Fortner, 2006). Other studies identified other 
challenges such as overcrowding of the curriculum (Pearson, 
Honeywood, & O’Toole, 2005) and the lack of resources and 
marginalization of EE in education (Summers, Childs, & 
Comey, 2005). Despite these challenges, EPPs have the 
potential to bring about significant change through the 
education of new teachers and professional development for 
practicing teachers in EE (UNESCO, 2012). 

In response to the challenges identified above, the North 
American Association for Environmental Education 
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(NAAEE) developed a set of national recommendations 
regarding the basic knowledge and skills educators need in 
order to provide high-quality EE. The Guidelines for the 
Preparation and Professional Development of Environmental 
Educators are designed to be applied: (a) within the context of 
EPPs; (b) to the professional development of educators in both 
formal and nonformal educational settings; and (c) to all 
educators who are responsible for the integration of EE into 
curricula and programs (NAAEE, 2004). The underlying 
assumption of the Guidelines is that teachers should have a 
working knowledge of what comprises environmental literacy 
and a clear understanding of how this knowledge can impact 
environmentally sound behaviors. 

Disinger and Roth (1992) defined environmental literacy 
as “the capacity to perceive and interpret the relative health 
of environmental systems and take appropriate action to 
maintain, restore or improve the health of those systems” 
(p. 2). The NAAEE Guidelines (2004) further states that 
environmental literacy depends on the learner’s ability to 
question, analyze, and interpret environmental issues; 
understand the processes and systems that make up the 
environment; and define, learn about, evaluate, and act on 
environmental issues, taking personal and _ civic 
responsibility. More recently, the Framework for Assessing 
Environmental Literacy (Hollweg et al., 2011) defines “an 
environmentally literate person as someone who, both 
individually and together with others, makes informed 
decisions concerning the environment; is willing to act on 
these decisions to improve the well-being of other 
individuals, societies, and the global environment; and 
participates in civic life” (p. 2). Accordingly, the NAAEE 
Guidelines describes an environmentally literate citizenry 
as individuals who understand and think in terms of 
systems and are capable of making well-informed decisions 
rooted in the belief that “humans can live with nature and 
act equitably toward each other” (p. 2). 

To further address the issue of teacher competency in 
environmental literacy, the NAAEE established the 
Preservice Teacher Advisory Council within its organization 
whose primary focus is the preparation of future teachers in 
EE. With support from the Council, a national research study 
was developed to investigate the current state of inclusion of 
EE into EPPs across the United States. This national study 
expands upon prior studies conducted over a decade ago 
(Heimlich, Braus, Olivolo, McKeown-Ice, & Barringer- 
Smith, 2004; Mastrilli, Johnson, & McDonald, 2001; 
McKeown-Ice, 2000) which reported the following 
limitations regarding the inclusion of EE in EPPs: (a) few 
EPPs had institutionalized EE into their curriculum 
frameworks, (b) EEPs varied greatly across the United States, 
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and (c) preservice teachers were not receiving the education 
necessary to effectively teach EE in preK—12 schools. 


Research Questions 

Because of the importance of including EE in schools, the 
need to adequately prepare teachers to teach about the 
environment, and the challenges of implementing EE into 
teacher education, the purpose of this national study was to 
address the following broad question: How are preservice 
teachers in the United States being prepared to teach 
primary and secondary students about the environment? In 
keeping with this purpose, we developed the Environmental 
Education Teacher Preparation Survey (EETPS). This 
survey explored the extent to which the NAAEE Guidelines 
have been integrated into programmatic decisions and 
practice, the factors that serve as supports or barriers to 
including EE in EPPs, and the EE resources being used. The 
format of the survey instrument contained items designed to 
specifically address five guiding research questions (RQ): 


¢ RQI: How is environmental education included in 
educator preparation programs across the United States? 

* RQ 2: What are the supports and barriers toward the 
inclusion of environmental education into educator 
preparation programs? 

* RQ3: How are the NAAEE Guidelines implemented in 
educator preparation programs? 

¢ RQ4: What are teacher educators’ perceptions about 
the effectiveness of educator preparation programs in 
preparing preservice teachers in EE? 

* RQ5: What EE resources are used in educator 
preparation programs? 


Methodology 

The EETPS consists of six parts designed to address the 
five research questions. Part I summarizes the intent of the 
research and solicits consent to participate in the study. Basic 
demographic information about participants and institutions 
is requested. Part II explores basic programmatic structures 
and requirements of EE within the EPPs, including state 
licensure options, course requirements, and degree options 
available (RQ1). Part II focuses on supports and barriers 
related to incorporating EE into EPPs (RQ2), and Part IV 
assesses teacher educators’ familiarity with the Guidelines 
and the role of the Guidelines in program development and 
assessment (RQ3). In Part V, participants self-reported on 
their perceptions of their EPPs’ effectiveness in EE 
preparation (RQ4), and in Part VI, they identified supporting 
EE resources and opportunities provided to preservice 
teachers in their programs (RQS). 
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Data Collection 

Descriptive research methodology was used to obtain a 
description of characteristics of current EPPs as reported in 
survey responses. As explained by Jackson (2011), surveys 
can be successfully used in research when “questioning 
individuals on a topic or topics and then describing their 
response” (p. 17). The survey responses were collected and 
aggregated by institution and disaggregated for each 
research question. 

Contacts within higher education institutions affiliated 
with an educator preparation program were obtained from 
the North American Association for Environmental 
Education (NAAEE) and the Council for the Accreditation 
of Educator Preparation (CAEP). The lists of contacts from 
the two organizations were sorted by institution and 
duplicates were eliminated. The invitation to complete the 
survey was initially emailed to 748 institutional contacts 
from across the United States. A second invitation was 
emailed to all contacts a month later. 

Section I of the survey reported responses from 211 
institutions of higher education in 37 states, Washington, DC, 
and Puerto Rico, indicating a 28% return rate (211 responses 
out of 748 contacts). Results indicated that 62% of the 
respondents were from public and 38% from private 
institutions of higher education. Responses came from 
institutions that reported multiple degree and program types 
that lead toward teacher certification: 86% of the responding 
institutions offer four-year undergraduate programs 
(baccalaureate degree), 40% have post baccalaureate 
programs (non-degree; licensure only), and 49% grant 
graduate degrees (master and/or doctoral programs). 

According to Nulty (2008), online surveys have an 
average response rate 23% lower than paper-based surveys. 
The paper-based EE survey conducted by McKeown-Ice 
(2000) had a response rate of approximately 62%. We 
acknowledge that our data set (n = 211) with a response rate 
of 28% (lower rate than the comparable McKeown-Ice 
study) could potentially contribute to a limitation to our 
findings. However, we ascertain that our response rate 
adequately represents a national probability sample of EPPs 
for an online survey in that we received responses from 211 
institutions, representing 37 states and 2 territories. We 
attribute the lower response rate due to the time at which the 
survey was administered. Initial surveys were emailed to 
contacts in June with the follow-up email sent in late July 
when some institutions of higher education are not in session. 
Study Context 

Because teacher education in the United States is 
decentralized, classroom teachers must be licensed by the 
state in which they teach. Regarding the discipline of EE, 
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only four out of SO states (Wisconsin, Pennsylvania, 
Kentucky, and Washington) have standards or licensure 
requirements related to EE in teacher education (Simmons, 
2014). Recognizing the importance of including EE in 
teacher education, the NAAEE established the Guidelines 
for the Preparation and Professional Development of 
Environmental Educators (NAAEE, 2004). These 
Guidelines provides recommendations about the basic 
knowledge and skills that both formal and nonformal 
educators need in order to provide effective EE instruction. 
Survey Development 

Following standard measurement criteria for developing 
valid measures, the items on the EETPS were subjected to 
multiple cycles of development, reviews, and revision. 
Content and construct validity of the survey were achieved 
through a review process involving three validity panels 
and a focus group of experts in EE and teacher education. 
The research team for this study was comprised of three 
university teacher educators with expertise in EE 
knowledge, research, and pedagogy. The EETPS was 
developed for this study through modification of items on 
previous national surveys relating to EE (Heimlich et al., 
2004; McKeown-Ice, 2000) and development of new items 
through examination of the literature related to EE and the 
NAAEE~ Guidelines. Specifically, the instrument 
development process involved the use of expert feedback 
with respect to content addressed and language of items 
within the survey. 

The first draft of the survey, developed by the research 
team, was initially reviewed by a panel of nine members of 
the NAAEE Preservice Advisory Council. This panel of 
teacher educators with expertise in EE examined the survey 
draft for content accuracy, clarity, and representativeness of 
items. Feedback from this panel was used to revise the first 
draft of the survey. 

Next, a focus group of 20 members of the NAAEE 
Preservice Advisory Council was administered the revised 
survey (Draft 2) at an annual meeting of NAAEFE. Focus 
group members were asked to provide oral feedback about 
the clarity of instructions; wording of items; time devoted 
to completing the survey; and response format, length, and 
content. The research questions of the study were revisited, 
and the feedback obtained from this discussion resulted in 
additional revisions to the instrument (Draft 3). 

Draft 3 of the survey was then sent for additional review 
by a second panel of two content experts in EE with 
extensive knowledge of the NAAEE Guidelines. Based 
upon their feedback, the survey was again revised (Draft 4). 
While Drafts 1, 2, and 3 were emailed to reviewers as Word 
documents, Draft 4 was loaded into an electronic medium 
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and field tested with a third panel of two experts in teacher 
education with limited expertise in EE for technical 
components, language congruency, sentence construction, 
and word usage. Panel members were given a link to an 
online survey, which the authors created using 
SurveyMonkey (2012) (http://www.surveymonkey.com). 
They identified areas where content assumptions or 
technology uses existed or terminology was ambiguous or 
confusing. This feedback was used to revise and finalize the 
EETPS: 
Data Analysis 

All responses from the EETPS were compiled in the 
SurveyMonkey platform. Using descriptive statistics, data 
collected from the survey were analyzed using a frequency 
distribution analysis of individual survey questions and 
expressed primarily as percentages. The survey consisted of 
six response options that varied based on the nature of the 
questions. These included (a) yes/no responses, (b) short 
answers, (c) check all that apply indicators, (d) multiple 
choice selections, (e) Likert scale ratings, and (f) multiple 
choice matrices. There were also opportunities for optional 
open ended responses in each section. Because these 
responses corroborated or slightly elaborated upon the 
answers to the closed questions, they were not coded but 
rather used to support survey question data and reported as 
direct quotes from respondents (Thomas, McColl, Priest, & 
Bond, 1996). Not all respondents addressed every question 
in the survey; data reported for each question indicate the 
number of responses for that particular survey item. 


Results and Discussion 

Research Question 1 — Inclusion of EE 
Part II of the survey explored the components of state 
licensure options, course requirements, and degree options 
included in the EPPs. Less than 15% of the respondents 
reported that their states offer some form of teacher 
licensure in EE: initial teacher licensure, 4%; endorsement, 
11%; certificate (non-teaching), 13%; advanced 
certification, 1% (graduate level) (see Table 1). These state 
licensure options are those used by CAEP for national 


Table 1 
State Credential Options in Environmental Education (n = 115) 
Yes No Do Not Know 
% (n) % (n) % (n) 

Initial Certification 4% (5) 70% (79) 26% (29) 
Endorsement 11% (12) 57% (65) 33% (37) 
Certificate 13% (15) 53% (60) 34% (39) 
Advanced 1% (1) 60% (68) 39% (44) 
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Table 2 
# Students in the Teacher Education Program Required to Take One or More 
Separate Courses in Environmental Education (n = 107) 


iy 69 No Do Not Know 
% (n) % (n) % (n) 
Early childhood 7% (7) 86% (86) 7% (7) 
Elementary 12% (2) 81% (83) 7% (7) 
Middle 7% (7) 86% (83) 7% (7) 
Secondary 10% (10) 82% (84) 6% (7) 
All level 2% (2) 87% (82) 11% (10) 


accreditation of EPPs. Due to the nonexistence of a teacher 
licensure or endorsement program in EE at the institutional 
level, 70% of the respondents indicated their institutions do 
not recommend students for state teaching licensure in EE, 
and 57% of the programs do not recommend students for 
endorsements in EE. 

Less than 15% of the responses indicated that preservice 
teachers are required to take a specific course in EE, 
regardless of licensure levels (early childhood, 7%; 
elementary, 12%; mid-level, 7%; secondary, 10%; all level, 
2%) (see Table 2). However, as reported in the survey, EE 
content is partially embedded into other courses within 
existing curricula for teacher education, occurring most 
frequently at the elementary licensure level (52%) (see 
Table 3). According to one respondent: “EE is basically 
embedded into one required science course, the elementary 
methods in math and science and with content area literacy 
in secondary ed. However, it probably gets more exposure 
through the local, regional, and state media as news breaks 
about drought, use of water, wind, heat, etc.” 

Research Question 2 — Supports and Barriers 

When identifying supports and barriers for including EE 
in their EPPs, participants were asked to rank provided 
factors on a 5-point Likert scale ranging from strong barrier 
(1) to strong support (5). Aggregate percentages reflect how 
each factor was perceived by respondents across the 


Table 3 
Programs with EE Content Partially Embedded into Teacher Education 
Coursework (n = 105) 


Yes No Do Not Know 
% (n) % (n) % (n) 
Early childhood 39% (35) 46% (41) 15% (13) 
Elementary 52% (49) 37% (35) 11% (10) 
Middle 38% (29) 47% (36) 16% (12) 
Secondary 44% (39) 42% (37) 14% (12) 
All levels 10% (6) 63% (38) 27% (16) 
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response options. While separated in Table 4, data are 
combined in the article discussion by clustering Major 
Support with Minor Support and Major Barrier with Minor 
Barrier. Overall, more barriers than supports were indicated 
for including EE in EPPs (see Table 4). 

One of the strongest supports reported was collaboration 
between teacher education departments and informal 
education sites that are proponents of EE (43%) such as 
local parks and nature centers. The survey results also 
indicated that more than one third of the respondents (41%) 
recognized that student interest has the potential as a 
support. Students can provide the catalyst for initiating an 
EE program at their institutions. Another support reported 
includes the diversity and availability of EE activity guides, 
curricula, and general resources for the use in curriculum 
planning and teaching (41%). In addition, an institutional 
mission that includes issues of environmental responsibility 
was cited by some survey respondents (35%) as being an 
important factor that supports the systematic integration of 
EE. When identifying the perceived barriers to including 
EE in the teacher preparation programs, 80% of the 
respondents indicated that allocated course time, or lack 
thereof, was a barrier. Other barriers that were identified 
included the lack of: (a) funding for an EE program (67%); 
(b) state teaching certification or degree requirements in EE 
(53%); (c) faculty interest in EE (53%); and (d) a 
systematic approach involving all stakeholders in infusing 
EE into the teacher preparation program (52%). 

While supports were evident, the data from this survey 
suggest there are more perceived barriers than supports for 
the inclusion of EE in EPPs across the country. One 
respondent summed up barriers at his/her institution to be: 
“(Lack of] budgetary resources, released time, commitment 
of fiscal and programmatic support at the university level.” 
Over half of the respondents (54%) identified that their state 
licensure guidelines (or lack of) is a major barrier for 
including EE in EPPs. 

Research Question 3 — Familiarity and 
Implementation of EE Guidelines 

When questioned regarding familiarity with the NAAEE 
Guidelines, 43% of the respondents indicated they had 
never heard of them, 32% were somewhat familiar with 
them, and 24% were very familiar (see Table 5). As one 
respondent indicated: “Never heard of EE standards. As I 
read them I realized we nonetheless do these things as good 
overall educators.” When asked about the extent to which 
the Guidelines were integrated into their EPPs, 6% fully 
integrate them for program development, and 3% fully 
integrate the Guidelines for programmatic assessment (see 
Table 6). 


Volume 117 (3-4) 


Table 4 


Supports and Barriers to Including EE in Educator Preparation Programs (n = 94) 


Major Barrier 


% (n) 
Collaboration between teacher 4% (4) 
education department and non-formal 
education agencies and organizations 
relevant to EE 
Student interest in EE 9% (8) 
Availability of EE activity guides, 2% (2) 
curricula and general resources for planning 
Institutional vision/mission 12% (11) 
Collaboration between teacher education 6% (5) 
and other departments 
Access to substantive, up-to-date EE resources 4% (4) 
Professional association guidelines and standards 10% (9) 
Administrative support in EE 20% (18) 
Educators’ knowledge related to the 20% (18) 
environment and environmental issues 
Educators’ preparation time 22% (20) 
Educators’ knowledge of EE methods 24% (22) 
Educators’ interest in EE 20% (18) 
A systematic approach (all stakeholders) 30% (27) 
to infuse EE 
State certification guidelines 34% (31) 
Funding 38% (35) 
Allocated course time 57% (52) 
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Minor Barrier 


% (n) 
10% (9) 


15% (14) 
17% (16) 


23% (21) 
19% (17) 


27% (25) 
17% (16) 
22% (20) 
27% (25) 


29% (27) 
27% (25) 
34% (31) 
22% (20) 


20% (18) 
29% (27) 
23% (21) 


Neither 
% (n) 


44% (40) 


36% (34) 
40% (37) 


31% (29) 
42% (38) 


42% (39) 
39% (36) 
42% (39) 
20% (18) 


41% (38) 
22% (20) 
23% (21) 
34% (31) 


35% (32) 
29% (27) 
12% (11) 


% (n) 
21% (19) 


32% (30) 
21% (19) 


26% (24) 
22% (20) 


15% (14) 
24% (22) 
9% (8) 
16% (15) 


5% (5) 
14% (13) 
11% (10) 
11% (10) 


8% (7) 
4% (4) 
1% (1) 


% (n) 
22% (20) 


9% (8) 
20% (18) 


9% (8) 
12% (11) 


11% (10) 
10% (9) 


17% (16) 


3% (3) 
14% (13) 
13% (12) 
3% (3) 


4% (4) 
0% (0) 
7% (6) 


Minor Support Major Support 


Research Question 4 — Effectiveness of Preparation 

When respondents were asked to self-report how they 
perceived the effectiveness of their overall EPP in preparing 
preservice teachers to teach EE (see Table 7), less than 10% 
stated their programs were Very Effective and less than 15% 
identified Moderately Effective. When asked how they perceived 
the effectiveness of their educator preparation program based on 
six listed factors of preparation, “Conveying environmental 
action strategies related to EE” ranked the highest for Not at all 
Effective (51%), and “Identifying local resources for EE” was 
ranked the highest for Very Effective (11%). 
Research Question 5 — EE Resources 

Given a list of EE resources as recommended by 
members of the NAAEE Preservice Advisory Council, 
participants were asked to indicate if each resource was Not 


Table 5 
Extent of Familiarity with Guidelines for Preparation and Professional 
Development of Environmental Educator (NAAEE, 2004) (n = 99) 


Never Somewhat Very 
Heard of Them Familiar Familiar 
% (n) % (n) % (n) 
You are familiar 43% (43) 32% (32) 24% (24) 


with the 
Guidelines 
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Used, Referenced, Integrated, or Do Not Know (see Table 
8). Overall, two-thirds of the listed resources were 
identified as Not Used or Do Not Know. “State or 
Government publications” were most often Referenced 
(28%) while “Local or regional materials and resources” 
were most often Jntegrated (36%) into the curriculum in 
EPPs. Considering international/national curricula, Project 
WET (33%), Project WILD (34%), and Project Learning 
Tree (PLT) (35%) were reported as the most often 
integrated resources; yet, one-fifth of the respondents 
reported that they Did Not Use or Did Not Know these EE 
resources. Additionally, 14 respondents listed several other 
resources that included Fish Banks, Investigating and 
Evaluating Environmental Issues and Actions (IEEIA), 
Energy Smart, Earth Gauge, and Lester Brown’s Plan B. 


Table 6 
Extent of Integration of Guidelines for Preparation and Professional 
Development of Environmental Educator (NAAEE, 2004) (n= 99) 


Do Not Somewhat Fully 
Integrate Integrate Integrate 
% (n) % (n) % (n) 
Programmatic 62% (59) 32% (31) 6% (6) 
development 
Programmatic 68% (64) 30% (28) 3% (3) 
assessment 
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Table 7 
Effectiveness of Teacher Preparation Program (n= 88) 
Not at All Somewhat Moderately Very 
% (n) % (n) % (nn) % (n) 
Conveying environmental content knowledge 31% (28) 47% (43) 13% (12) 9% (8) 
Conveying instructional methods related to EE 34% (30) 41% (37) 15% (14) 10% (9) 
Educating about environmental issues 25% (21) 52% (47) 14% (13) 11% (9) 
Graduating students with a basic level 34% (30) 49% (44) 8% (7) 9% (8) 
of environmental literacy 
Conveying environmental action 51% (46) 33% (30) 19% (9) 6% (6) 
strategies related to EE 
Identifying local resources for EE 30% (27) 42% (37) 16% (15) 11% (10) 
Effectiveness of EE preparation overall 35% (31) 50% (44) 8% (7) 7% (6) 





Discussion 

Thirty years ago, Wilke (1985) reported that “teacher 
education programs in environmental education remain 
relatively scarce and poorly developed” (p. 1). Despite 
research studies indicating that EE can provide positive 
student outcomes in academic achievement, critical thinking, 
motivation, and engagement (Knapp & Benton, 2006; Smith- 
Sebasto & Semrau, 2004; Stern, Powell, & Ardoin, 2008), 
our survey results indicate that preservice teachers receive 
limited exposure to EE in their teacher preparation programs. 
These results mirror results from earlier studies completed by 
McKeown-Ice (2000), Lane, Wilke, Champeau, and Sivek 
(1994), and Smith-Sebasto and Smith (1997). In fact, the 
responses for overall effectiveness in this survey indicated 
over 75% of the respondents rated their programs as Not at 
all Effective or Somewhat Effective, while the study done by 
McKeown-Ice stated 66% of the respondents rated their 
programs poor or adequate. 


There still exist a limited number of states that offer 
teacher licensure in EE. Consequently, few universities 
require specific coursework or competencies in EE. As a 
result, the implementation of EE in teacher education across 
the nation is: (a) generally not institutionalized or 
systematic, (b) varies across programs and states, (c) tends 
to be in more elementary than secondary education 
programs, and (d) if present, is typically most often infused 
into existing science methods courses (Mastrilli, 2005; 
Powers, 2004; Simmons, 2014). If EE was identified in the 
survey as a component of an EPP, the level and design of 
implementation varied greatly across the nation. As one 
respondent lamented, “I wish we could do more. As it is, I 
sneak EE in my courses.” 

EPPs are mandated by state legislatures to include 
specific content and pedagogical courses, oftentimes within 
a limited number of total credit hours. Eighty percent of the 
survey respondents indicated that state requirements leave 


Table 8 
EE Resources (n = 79) 

Not Used 

% (n) 
Early childhood PLT 41% (37) 
Environmental issues forums 53% (42) 
Facing the future 40% (36) 
Growing up WILD 36% (28) 
Issue investigation and evaluation 36% (28) 
Env. issues 41% (37) 
Local or regional materials and resources 24% (19) 
Nature scope 39% (31) 
Project learning tree 24% (19) 
Project WET 23% (18) 
Project WILD 22% (17) 
Think earth 57% (45) 
World Watch Institute 49% (39) 
Zero population growth 42% (33) 
State or government 25% (20) 


Referenced Integrated Do Not Know 
% (n) % (n) % (n) 
13% (10) 14% (11) 37% (29) 
9% (7) 11% (9) 37% (29) 
18% (14) 9% (7) 38% (30) 
22% (17) 16% (13) 38% (30) 
24% (19) 23% (18) 27% (22) 
15% (12) 8% (10) 34% (27) 
21% (16) 39% (31) 25% (20) 
21% (16) 10% (8) 37% (29) 
25% (20) 38% (30) 23% (18) 
27% (22) 37% (29) 25% (20) 
32% (25) 38% (30) 22% (17) 

8% (6) 3% (2) 43% (34) 
16% (13) 6% (5) 38% (30) 
16% (13) 14% (11) 38% (30) 
28% (23) 22% (17) 27% (22) 


a 
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little to no room for support or elective courses such as EE 
in degree plans. As indicated by one respondent: “Given 
state DOE [Department of Education] and NCATE 
accreditation course requirements, there is no longer any 
room to require EE coursework in our teacher prep 
programs. Instead, mandated courses such as reading and 
ESL [English as Second Language] have replaced EE 
courses.” 

The findings in this study suggest that EE is not 
recognized as a discipline. This lack of recognition 
indirectly provides a barrier as compared to other 
disciplines that do require state licensure. As stated by one 
respondent: “EE is not listed in the state standards and thus 
we do not need to offer EE as a part of our program for all 
students.” This could account for 52% of the survey 
respondents identifying a lack of a systematic approach as a 
barrier for infusing EE into their EPPs. This response 
supports existing literature that suggests a systematic 
approach at a university-wide level is required to impact 
change at the departmental and/or program level (Ferreira, 
Ryan, & Tilbury, 2006). According to one respondent: “We 
have no systematic attention to EE and faculty are pulled in 
many ways to focus content on legislated requirements for 
teacher preparation (e.g., civic and local government, 
technology, assessment).” As expressed by another 
respondent: “We just do not have a systematic approach. I 
believe we have the resources, but there are so many other 
standards and requirements bearing down on programs that 
we find it difficult to integrate everything into a compressed 
curriculum.” 

According to Greenwood (2010), “It is possible to work 
within the state-regulated system to create change, but first 
individuals and groups need to build relationships, 
demonstrate credibility, and then respond strategically to 
whatever openings exist” (p. 151). Systemic change may 
occur if committed teacher educators knowledgeable in EE 
engage in the development of strong partnerships and 
collaborations with other educators in teacher education 
and across campus, practicing teachers, nonformal 
educators, and policy makers. These partnerships can help 
broaden preservice teachers’ knowledge about local 
resources and work with state teacher licensure agencies 
toward the inclusion of EE in teacher education. 

Plevyak, Bendixen-Noe, Henderson, Roth, and Wilke 
(2001) wrote that “due to the interdisciplinary nature of EE, 
including environmental education in preservice teacher 
preparation programs is a challenging task” (p. 28). This 
challenge still exists today for the integration of the 
concepts and perspectives from the sciences with those 
from the social sciences in addressing solutions to 
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environmental problems. Jaritz (1996) also recognizes this 
existing dichotomy where “isolated approaches to 
environmental education, which are tailored to the purposes 
of particular subjects, make it extremely difficult to prepare 
future teachers for their task to realize environmental 
education in an inter-disciplinary context’ (p. 61). 
However, according to our survey, this seems to be the 
most frequent strategy for infusing EE into the teacher 
preparation programs, with the elementary science methods 
course as the course targeted the most: 

Almost two decades ago, McKeown-Ice (2000) 
recommended that national standards or guidelines for the 
preparation of preservice teachers in EE should be 
developed. As a consequence, the NAAEE Guidelines for 
the Preparation and Professional Development of 
Environmental Educators now exist. However, despite the 
existence of these national Guidelines for over a decade, 
few teacher educators are aware of them and/or do not use 
them for program development or assessment. Teacher 
educators should consider incorporating the Guidelines into 
their programs and course work for this purpose. 

We consider the lack of familiarity with the NAAEE 
Guidelines to be an indirect barrier to the inclusion of EE in 
EPPs. The Guidelines provide the competencies, 
background, resources, and information necessary for 
teacher educators preparing environmental educators. The 
fact that almost half of the respondents were not familiar 
with these national standards is cause for concern. One of 
the goals of NAAEE is to effectively disseminate 
information about the Guidelines nationally, and in order to 
do so, members of NAAEE provide training across the 
country. The results of this survey indicate that a renewed 
focus on familiarizing teacher educators with the 
Guidelines is needed. In addition, further research is needed 
regarding the impact of the integration of the Guidelines 
into an EPP on preservice teachers’ attitudes about and 
knowledge of the environment. 

While it is encouraging to see that some EPPs are 
utilizing specific resources, it is problematic that easily 
accessible EE resources and materials are not being utilized 
by many of the respondents. The list provided in the survey 
included 16 known EE resources in the field, as 
recommended by experts in EE and determined through the 
survey development process. Given that each of the 16 
options listed was integrated by at least one respondent and 
respondents added only a few additional resources to the 
list, we feel the list was reflective of the most frequently 
used EE resources available to teachers. Established EE 
curricula should be introduced to preservice teachers across 
their program of study. These curricula can be integrated 
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into existing teacher education courses as examples of the 
integration of subject areas and inquiry-based instruction 
that promote active learning. 

An emphasis on student-led campus and community 
projects is another way to integrate EE into existing 
curricula because students generally focus on local 
problem-solving situations (Krizek, Newport, White, & 
Townsend, 2012). Studies now show that two-thirds of 
prospective college freshmen look at campus green 
rankings as a factor in college choice (Pryor, Hurtado, 
Saenz, Santos, & Korn, 2007). However, to ensure 
sustainability, the infusion of EE into an EPP needs to go 
beyond ad hoc activities such as a campus wide recycling 
program and be recognized as an important component 
across curriculum, research, service, operations, and 
campus life (Ferrer-Balas et al., 2008; Krizek et al., 2012; 
Velazquez, Munguia, & Sanchez, 2005). 

Our survey results indicate more needs to be done in 
preparing future teachers for the responsibility of 
developing environmentally literate citizens. The work by 
Plevyak et al. (2001) suggests that “successful 
implementation of EE includes teacher preparation” (p. 36), 
and our survey results indicate more still needs to be done 
in preparing future teachers for the responsibility of 
developing environmentally literate citizens. We believe a 
critical step at reaching environmental literacy for all is to 
equip preservice teachers with the necessary skills, 
knowledge, attitudes, and concern of environmental issues 
that would enable them to implement EE in their future 
classrooms. To enhance future teachers’ EE competencies, 
EPPs must include the systematic implementation of EE 
(Van Petegem, Blieck, & Boeve-De Pauw, 2007) and 
integrate EE as a part of their programs and curricula rather 
than an occasional add-on (Heimlich et al., 2004). 

Yavetz, Goldman, and Peier (2014) argue that “all 
student teachers should receive appropriate preparation in 
this field... EE [should] not be limited to science 
disciplines; rather it should be included in all teacher 
education programs” (p. 370). Preservice teachers should 
be provided with opportunities to learn and practice 
effective instructional strategies on how to share these 
experiences with children. We know educating children in 
classrooms will potentially increase the environmental 
literacy of all citizens. To foster the development of these 
environmentally literate citizens, we need appropriately 
prepared teachers. This preparation must begin in the EPPs. 
As Ashmann and Franzen (2015) state: 


If teachers are not adequately prepared to teach our 
youth about the environment, then our hopes of the next 
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generation being able to solve these problems has 
incurred a serious setback. Teachers who are well 
prepared to teach about the environment can have a 
lasting impression upon their students. (p. 2) 


Recommendations 

As researchers in EE, we have concluded from our own 
research with preservice teachers that many come to the 
university with simplistic perceptions of the environment 
(Moseley et al., 2014). These perceptions or beliefs may be 
founded and influenced in a number of ways as a result of 
an individual’s education, the media, and personal, 
everyday lived experiences. We also recognize that 
behavior and perceptions can be difficult to change. 
Furthermore, we are aware of the research in teacher 
professional development that includes effective strategies 
such as structured and collaborative study groups, 
coaching, and immersion in inquiry (Boyle, While, & 
Boyle, 2004). However, these strategies can sometimes be 
difficult for EPPs to implement as a short-term intervention. 
Based on the results from this survey, we recommend that 
teacher educators, in promoting best practices in 
professional development, should form partnerships with 
informal EE organizations, capitalize on student interest, 
and utilize available EE curriculum materials. 

The foundation of the EE program within our EPP rests on 
the belief that environmental knowledge alone is not 
sufficient. In order to bring about behavior change, 
preservice teachers also need to have a variety of experiences 
outdoors over time in order to become environmentally 
sensitive. In addition, they need opportunities within the 
higher education setting to engage in focused and reflective 
explorations of environmental issues. Our prior research 
suggests the majority of our teacher candidates continue to 
adhere to an object conception of the environment despite 
the increasing exposure to EE provided throughout their 
teacher preparation program (Moseley et al., 2014). 
Therefore, further research is needed to investigate which 
strategies are more effective in developing preservice 
teachers’ environmental literacy and challenging their 
assumptions and beliefs. Determining effective ways on how 
to include an interdisciplinary approach to EE in programs 
that may conflict with state policies is another critical area 
needing further research. 

Collectively, teacher educators need to provide a focused 
and consistent effort toward raising awareness of the critical 
need for EE in EPPs if any progress is going to be made in 
this area. EE needs to become an integral component of every 
EPP. As Papadimitrious (2004) states, “educating students 
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about such issues [like EE] is also a challenge for educators 
because the traditional didactic strategies are inappropriate 
and so new innovative instructional strategies and techniques 
should be invented which means that a vast area for research 
is opened” (p. 306). This applies to teacher educators as well. 
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To make progress toward ambitious and equitable goals for students’ mathematical development, teachers need 
opportunities to develop specialized ways of knowing mathematics such as mathematical knowledge for teaching (MKT) 
for their work with students in the classroom. Professional learning communities (PLCs) are a common model used to 
support focused teacher collaboration and, in turn, foster teacher development, instructional improvement, and student 
outcomes. However, there is a lack of specificity in what is known about teachers’ work in PLCs and what teachers can 
gain from those experiences, despite broad claims of their benefit. We discuss an investigation of the work of secondary 
mathematics teachers in PLCs at two high schools to describe and explicate possible opportunities for teachers to develop 
the mathematical knowledge needed for the work of teaching and the ways in which these opportunities may be pursued or 
hindered. The findings show that, without pointed focus on mathematical content, opportunities to develop MKT can be 
rare, even among mathematics teachers. Two detailed images of teacher discussion are shared to highlight these claims. 
This article contributes to the ongoing discussion about the affordances and limitations of PLCs for mathematics teachers, 


considerations for their use, and how they can be supported. 


Recommendations for mathematics teacher professional 
development (PD) assert the need for a clear focus on 
teachers’ mathematical content knowledge and _ its 
connection to the work of teaching, especially in the 
context of new standards and the resulting demands on 
teachers’ work (Marrongelle, Sztajn, & Smith, 2013). An 
increasing number of schools and districts have worked to 
develop professional learning communities (PLCs) to 
support teacher development and promote instructional 
change and positive impacts on student achievement 
(Mindich & Lieberman, 2012). With PLCs, teachers are 
provided with time and other resources to collaboratively 
discuss instruction, common planning and assessment, and 
student data. While use of PLCs is in line with goals for 
ongoing, collaborative, and practice-based PD for teachers 
(Bausmith & Barry, 2011), the widespread use of PLCs is 
at a “crossroads” (Talbert, 2010) due to lack of clarity 
around their purpose and benefits and irregularity in their 
implementation. 

In this article, we look at a part of that dilemma. 
Specifically, we question the extent to which teachers’ 
work in PLCs—specifically when the PLCs are teacher-led 
and operate without researchers or skilled teacher leaders— 
reliably provides opportunities to develop mathematical 
knowledge. Attention to how teachers may develop their 
knowledge of the subject matter they teach is not regularly 
addressed in the literature on PLCs, despite broad claims of 
PLC’s benefits on content knowledge (Bausmith & Barry, 
2011). We present findings from a study investigating the 
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ways in which opportunities emerged for secondary 
teachers in PLCs to develop “mathematical knowledge for 
teaching” (MKT; Ball, Thames, & Phelps, 2008). We 
highlight hurdles that limited these opportunities and 
propose that some of these hurdles may ultimately be 
limitations of the broad potential of PLCs that should be 
considered in practice. 


Background on PLCs 

Across an extensive body of research and literature 
reviews, such as those from Fulton, Doerr, and Britton 
(2010) as well as Vescio, Ross, and Adams (2008), PLCs 
are broadly defined as groups of teachers who meet 
regularly and over an extended time to focus on issues of 
student learning in local contexts. DuFour and his 
colleagues (DuFour, 2014; DuFour, DuFour, & Eaker, 
2008)—whose ideas are commonly used to frame teacher 
collaboration in schools and districts—highlight a need for 
shared goals, vision, and values as well as an orientation 
focused on continuous improvement and student learning. 
Others such as McLaughlin and Talbert (2008) and 
Mindich and Lieberman (2012) similarly define PLCs, 
while also emphasizing the feature of shared or 
deprivatized practice by engaging in activities such as 
analyzing student work, designing and critiquing lessons 
and activities, and developing and examining data from 
common assessments to measure progress. 

The scope of a PLC in terms of its participants varies 
across the literature. For example, PLCs may refer to a 
group of teachers focused on the same grade level or course 
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(Mindich & Lieberman, 2012). These groups may develop 
on their own or as part of a school-wide effort. PLCs may 
be comprised of teachers from across schools, with the 
support of researchers or other teacher leaders and as part of 
a larger PD effort (Brodie, 2014). A PLC may also refer to 
a whole school and other stakeholders (DuFour, 2014; 
DuFour et al., 2008). In our work, we focus on PLCs as 
small groups of teachers that interact around common 
courses that were organized as part of a district-wide effort 
to promote teacher collaboration and support broad impacts 
on teaching and learning. We discuss this context in more 
detail in a section that follows. 

PLCs or other models of teacher community have been 
widely used as a feature in the design of mathematics PD and 
the impacts of these settings have been at the focus of research 
studies (Borko, Jacobs, Koellner, & Swackhamer, 2015; 
Brodie, 2014; van Es & Sherin, 2010). Across the body of 
work on PLCs, teachers’ participation in these communities 
has been shown to have positive impacts on teachers’ attitudes 
toward collaboration (Mindich & Lieberman, 2012); on 
teachers’ mathematical content knowledge, specifically 
around students’ thinking (Brodie, 2014; van Es & Sherin, 
2010); and on students’ achievement (DuFour, 2014), While 
the benefits of PLCs are wide-ranging and widely reported, 
they are often based in self-reports from practitioners with less 
known about how practices of teachers’ collaboration in PLCs 
support particular gains (Bausmith & Barry, 2011). 


Theoretical and Conceptual Framework 

The idea of PLCs builds from the sociocultural lens of 
communities of practice (Wenger, 1998). Teaching is a 
cultural activity comprised of the understandings, tools, and 
activities shared by the collective of practitioners, making it 
less about what individual teachers do and more about 
meanings that are negotiated collectively (Lampert, 2010). 
As mathematics teachers collaborate while situated in 
authentic work and contexts, the shared knowledge, 
experiences, and other resources of the group support 
development. This conception of learning in, from, and for 
teaching positions PLCs as a potential vehicle for teacher 
development. This theoretical perspective also has 
implications on how knowledge—particularly the way one 
knows the content they teach—is defined and how such 
knowledge is developed. 
Developing Teachers’ MKT as a Goal for PD 

While many studies and reports attribute participation in 
PLCs to the development of content knowledge, there is not 
consensus in what is meant by content knowledge nor how 
PLCs specifically support that development. Recent research 
on teacher PD (Borko et al., 2015; Elliott et al., 2009; 


116 


Lesseig et al., 2016) addresses this problem by specifying 
MKT as the content knowledge goal for mathematics 
teachers. Ball and her colleagues (2008) describe MKT as 
the mathematical knowledge-in-action needed to carry out 
the mathematical work of teaching—including analyzing 
and modifying tasks, choosing appropriate examples, and 
evaluating the correctness of and reasoning underlying 
students’ claims. While pedagogical skill is involved with 
these tasks, as well as knowledge of curriculum and students, 
teachers use their disciplinary knowledge in a way that is 
unique to the work of teaching, giving more specificity to the 
idea of “knowing mathematics deeply.” The construct of 
MKT has continued to be developed (Hoover, Mosvold, 
Ball, & Lai, 2016) and instruments to measure MKT have 
been designed and tested (Kaarstein, 2014). MKT has been 
linked to quality of mathematics instruction (Hill, Umland, 
Litke, & Kapitula, 2012) and student achievement (Kersting, 
Givvin, Thompson, Santagata, & Stigler, 2012), thus 
warranting its position as an important and viable way to 
focus the otherwise vague consideration of teachers’ 
mathematical content knowledge. 

Due to the nature of MKT, it is not something learned in 
college mathematics courses (Youngs & Qian, 2013), nor is 
it developed or used in other professions (Suzuka et al., 
2009). MKT is also not something teachers necessarily pick 
up in the context of their day-to-day work on the job. 
Efforts have been made to understand the contexts and 
supports that foster the development of MKT, including 
specialized tasks (Suzuka et al., 2009) and PD models that 
facilitate teachers’ engagement with mathematics (Borko 
et al., 2015; Elliott et al., 2009). As a result, what it looks 
like to do mathematics with teachers should be different 
from students’ work on mathematics in a classroom, 
resulting in a complex nature of teachers’ collective work 
around mathematics (Elliott et al., 2009). 

We take up these ideas from the literature and consider 
MKT to be a productive way to frame goals around 
teachers’ mathematical development, including in 
discussions about PLCs. The work of Borko and her 
colleagues (2015) as well as others highlights the way in 
which teachers’ MKT can be purposefully supported and 
developed in PLCs and other collaborative structures. 
These examples are powerful, but are insufficient as the 
only evidence to make claims about the benefits of all types 
of PLCs, even those that are comprised only of 
mathematics teachers but have different origins or supports. 


Research Questions 
With the study we describe in this article, we addressed a 
need to further understand the way in which the 
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collaborative structures of PLCs—as seen in a wider variety 
of uses and conceptualizations—may serve in the 
development of MKT or the additional supports and 
considerations that might be needed to support such 
specific goals. Further understanding of these opportunities 
and constraints add to a developing sense of how to 
structure, support, and rely on teacher collaboration in 
school-based sites as a form of professional development. 
In particular, we pursued the following questions through 
an investigation of the interactions of two teams of teachers 
operating as PLCs: 
1. To what extent do teachers’ discussions focus on 
mathematics content in PLCs? 
2. How do teachers’ ways of working and collaborating 
around mathematics-focused discussions foster or 
constrain opportunities to develop MKT? 


Methods 
Study Context 

The study took place in a large, urban school district in 
the southeastern United States that implemented a particular 
model of PLCs from DuFour and colleagues (2008) to 
structure teacher collaboration. This effort was spurred by a 
local school and business consortium focused on improving 
high school graduation rates. At the time of the study, this 
effort was in its third year of implementation. In high 
schools, many teachers were organized in subject-specific 
teams to plan common lessons, investigate current issues, 
and evaluate and make decisions about their practice using 
student data. Each team assigned a teacher to serve as the 
leader, tasked with facilitating the group’s meetings and 
attending monthly workshops with team leaders across the 
district. Based on research and other reports, such as that 
from Mindich and Lieberman (2012), this context of what 
PLCs come to mean in schools is common, thus making the 
investigation we report on here of broader importance. 

A focus on two PLCs. Two teams of mathematics 
teachers participated in the study. Each team was housed at 
a separate, large high school in the district, what we will 
call Brantley High School and Elmwood High School (all 
school and teacher names used are pseudonyms). These 
two teams were chosen based on their response to a call for 
participants in a small study regarding the use of a PLC 
model in the district, with a focus on high school 
mathematics teachers. From the teams across the district 
that responded to this call, the teams from Brantley and 
Elmwood were chosen as the two that would be studied 
due, in part, to similarities between the teams and the 
schools. Both teams were organized to focus on Algebra I 
classes within their broader mathematics departments. The 
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population at each school consisted of students from across 
the district, including students from the urban center of the 
district, resulting in socioeconomic and racial diversity in 
both schools. Each school’s mathematics department was 
comprised of more than 15 teachers (split up across 
multiple PLC teams). Both schools implemented a late start 
one day per week to allow for weekly, 45-minute meetings 
for each PLC throughout the school year. All teachers in 
each group were expected to spend this time in their 
groups, although a variety of factors (including school 
related factors such as student organizations) would result 
in occasional absences. 

This form of purposive sampling was a choice made by 
the researchers to limit some of the variation between the 
teams while potentially offering initial insight into 
similarities and differences of their collaborative practices. 
We did not make these choices to strive toward larger scale 
trends or unwarranted generalizations arising only from 
these two cases. Instead, we set out to construct images of 
these teams’ collaborative practices as they pertain to 
mathematics and mathematics teaching. Furthermore, we 
aimed to construct those images with detail that allowed for 
the emergence of further questions and the consideration of 
these images in other contexts. 

Brantley High School. The PLC that participated from 
Brantley High School consisted of five teachers—Hillary, 
Kate, Samantha, Luke, and Vincent. Four of the teachers 
(Hillary, Kate, Samantha, and Luke) had five or fewer years 
of teaching experience, and all five teachers had between 
two and four years of experience teaching Algebra I. Two 
of the teachers (Luke and Vincent) worked in other 
professions before starting their teaching career. Kate, 
Luke, and Samantha were part of the original Algebra I 
team at the school (in its third year), with Luke having 
served as the leader for the first two years. Hillary and 
Vincent were new to the team for the year of this study. 
Samantha assumed the role of team leader, selected by 
group consensus. As team leader, she was tasked with 
setting the agenda and moderating the group’s talk and 
time. However, Luke continued to attend the district-wide 
Algebra I team leader meetings as he did during the 
previous year. This division of labor highlights the different 
ways in which a PLC like the ones highlighted in this 
article can be led. Furthermore, during this time, Kate was 
acting as the mathematics department chair, which brought 
a different leadership influence into the group. 

Elmwood High School. The PLC that participated in 
the study from Elmwood High School consisted of four 
teachers—Jackie, Sabrina, Pam, and Doug. These teachers 
all taught an Algebra I course specifically serving students 
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who have previously taken and failed the course. Three of 
the four teachers (Jackie, Sabrina, and Doug) had taught for 
at least 10 years, although the team was newly organized 
around a focus on Algebra I. Two of the teachers (Jackie 
and Doug) had taught Algebra I for 13 years each but the 
other teachers were in their first year of teaching the course. 
Additionally, those teachers (Sabrina and Pam) were in 
their first year at Elmwood, having each previously taught 
at other schools. Jackie served as the team leader (and 
attended the district’s Algebra I team leader meetings) and 
was also in the process of completing her administrative 
internship as part of a graduate program. Doug was in the 
early stages of taking coursework part-time as part of a 
doctoral degree program in mathematics education. 
Data Collection 

The primary data used to pursue our research questions 
were records of each team’s interactions, specifically audio 
recordings of PLC team meetings and field notes completed 
by the first author. Over the course of the first semester of 
the school year, each team was observed four times during 
their regular weekly meeting time. Two of the observations 
of each team occurred in consecutive weeks to provide the 
opportunity to see teacher interactions in a sequence of 
meetings, with the other two observations spread over other 
times of the semester. The observations were non- 
participant in nature, with the observer not contributing to 
the discussions. We did not see the observer’s presence at 
the meetings as a limitation in terms of how it may have 
affected the substance of the interactions observed. In part, 
this is because the teachers did not know in specific detail 
what the researcher was attending to in his observations. 

The observer’s field notes kept a record of attendance at 
meetings, seating arrangement, the general agenda and flow 
of the meeting, and any observations on_ teachers’ 
interactions that would be relevant to the study. Due to this 
last point, some notes were more interpretive (e.g., noting 
agreement or conflict between teachers or noting potentially 
interesting mathematical moments or discussions). The 
field notes were reviewed with participating teachers 
individually as a form of member checking to ensure their 
validity relative to participants’ perspectives. 
Data Analysis 

We conducted analyses in three steps: (1) annotation, (2) 
coding, and (3) interpretation. For annotation, the audio 
recordings were segmented into two-minute intervals. 
Within each interval, a written record was kept of the 
events of the meeting through a descriptive narrative 
including any notable direct quotes. Using these 
annotations as a basis, we applied two types of codes to 
each interval in pursuit of our first research question. First 
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we coded each interval for the substance of the focus of the 
discussion (what we called “substance” codes; see Figure 
1). Substance codes were developed in a collaborative and 
iterative process (Smagorinsky, 2008), starting with both 
authors collaboratively developing and applying a list of 
substance codes based on the annotations of one group 
meeting from each team. The first author then 
independently coded the remaining meeting annotations, 
keeping a record of new or revised codes and other notes 
(e.g., expanding or combining codes). Both authors 
revisited the coding and notes to come to agreement and 
resolve any issues. 

Next we coded whether or not the topic of the discussion 
during each interval included focus on or reference to 
mathematics content (what we called the “math focus” 
code). We applied one of three codes to each interval: no 
mathematics, mathematics reference, and mathematics 
discussion. The “no mathematics” code was used when the 
interval involved no mention of specific mathematics topics 
or concepts. The “mathematics reference” code was used 
when specific mathematics topics or concepts were 
mentioned during the interval, although it was deemed to 
be in passing or only at a surface level. For example, when 
discussing an item on a shared assessment, teachers may 
have referred to the content of the item (e.g., scalar 
multiplication of matrices) and made a brief comment about 
the content (e.g., scalar multiplication of matrices seems to 
be easier for students than matrix subtraction; absolute 
value is no longer in the standards for the course). Finally 
the “mathematics discussion” code was used when there 
was interaction involving multiple people about specific 
mathematics content. From our data, these moments 
included discussions of where a particular topic should be 
in a sequence (and why) and how to teach a particular topic 
(and why). As with the substance codes, we applied “math 
focus” codes collaborative and individually, meeting to 
confirm consistency of coding and to address questions and 
resolve disagreements. 

The final step of our analyses (i.e., “interpretation’’) 
supported our inquiry into our second research question and 
focused on the “mathematics discussions” of each group. 
For this, we started by taking the two-minute intervals 
coded as “mathematics discussion” and combining them 
with related, sequential intervals, forming what we called 
“instances.” These instances captured complete segments of 
discussion in the PLCs that we felt were candidates for 
group interactions that could support the development of 
MKT. We collaboratively examined the audio records, and 
field notes from these instances to evaluate the way in 
which opportunities to develop MKT were fostered or 
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Substance Codes Examples 
Data analysis e 


Discussion and analysis of assessment data 


e Analysis and interpretation of individual assessment item 
e Analysis of student strategies 


Assessment design and logistics 


General discussion of design of assessments 


Identifying specific items to include on/as assessments 

Discussion of when to give various assessments, 

including common assessments from school or district 
e Discussion of use of pre-assessments and quizzes 


Policies, norms, and procedures e 


Discussion of expectations or policies around test and 


quiz retakes 


Unit and lesson planning 


Discussion of the assigning and/or grading of homework 
Discussion of other grading procedures and policies 


Discussion of content standards 

Sequencing of topics within and across units 

Sharing curricular resources 

Sharing instructional ideas and strategies, such as guided 


notes, procedures, technology activities, and approaches 
to teaching particular topics 


Other 


Announcements 
School logistics and issues 
Student issues (e.g., attendance, behavior) 
istics, such as general agenda- and goal-settin 


Group logi g g g g 


Figure 1. List of “substance” codes and examples of such discussion foci. 


constrained and to describe the apparent factors and other 
themes influencing the work. We documented _ this 
collaborative interpretation through analytic notes, which 
were then used to look across all relevant instances to 
identify themes and trends. 
Findings: Opportunities for Teachers’ Mathematical 
Development During PLC Meetings 

In this section, we address our two research questions, 
beginning with findings related to the mathematical focus 
of each team’s discussion, then continuing with more 
detailed discussion of potential opportunities for teachers to 
develop MKT in the context of PLCs. 


Research Question 1 

Table 1 shows a breakdown of each team meeting in 
terms of total number of intervals and the number of 
intervals assigned each of the five substance codes. This 
breakdown shows how the focus changed across meetings 
and differed between the two teams. A striking trend was 
the extent to which the Elmwood PLC spent discussing 
topics we categorized as “other’—such as student issues 
and other broad school issues. When considering the 
general purposes of teacher collaboration in this district— 
planning common lessons, investigating current issues, and 
evaluating and making decisions about their practice using 


Table 1 
Inventory of Substance Codes Assigned to Meeting Intervals 

Assessment 
Site and Meeting Total Data Design and Policies, Norms, Unit and Lesson 
Number Intervals Analysis Logistics and Procedures Planning Other 
Brantley #1 pa 6 6 6 3 2 
Brantley #2 Ze 7 3 3 2 4 
Brantley #3 2 5 5 0 6 7 
Brantley #4 22 0 2 fh 8 5 
Elmwood #1 17 1 i 0 6 3 
Elmwood #2 23 2 3 2 7 9 
Elmwood #3 oe 6 3 0 0 13 
Elmwood #4 20 2 3 2 2 11 
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Table 2 
Inventory of Meeting Intervals Organized by Substance and Math Focus Codes 
Assessment Policies, Unit 
Site and Math Data Design and Norms, and and Lesson 
Focus Codes Analysis Logistics Procedures Planning Other 
Brantley (90 total intervals) 
No mathematics 6 13 16 5 18 
Mathematics reference 10 3 0 10 0 
Mathematics discussion 2 0 0 7 0 
Elmwood (82 total intervals) 
No mathematics 4 14 4 5 36 
Mathematics reference 4 2 0 Z 0 
Mathematics discussion 3 0 0 8 0 





student data—it appears that these “other” topics were, in 
fact, off-topic. 

Attention to these broad purposes for PLCs in the district 
also illuminated two other trends that show differences 
between these two teams. From the data in Table 1, as well 
as from the audio recordings and field notes, it was evident 
that the Brantley teachers were focused on the use of 
assessment data. The team spent a lot of their time 
analyzing data from shared assessments, talking about 
upcoming assessments (both design and logistics), and 
discussing policies and norms around topics such as 
grading. Conversely, the Elmwood teachers appeared to be 
more focused (when not discussing “other” topics) on 
sharing instructional ideas and common planning. In 
addition to the counts of coded intervals, Jackie’s repeated 
reminders for teachers in the group to bring instructional 
ideas to share also signaled this apparent focus. 

We were ultimately more interested in the extent to which 
each team’s focus was on mathematics, although the 
substance codes allowed us to consider the types of PLC 
discussions provided the best opportunities for mathematical 
focus. Table 2 provides these data, considering the math focus 
of intervals across all of each team’s meetings, broken down 
by substance code. Across both teams, discussion intervals 
coded as “policies, norms, and procedures” or as “‘other’’ did 
not include any mathematics reference or discussion and 
group discussions about “assessment design and logistics” 
only included occasional reference to mathematics. While 
these types of collaborative interactions may serve some need 
or goal (regarding PLCs or coming from the teachers 
themselves), they were not sites for developing MKT due to 
their lack of mathematical specificity. 

The other two substance codes—‘“data analysis” and 
“unit and lesson planning’—provided the sites in which 
teachers’ interactions were more focused on mathematics. 
For the Brantley PLC, a majority of the intervals coded as 
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either “data analysis” or “unit and lesson planning” were 
also coded as “mathematics reference” or “mathematics 
discussion” (as opposed to “no mathematics”). Of those, 
many were coded as “mathematics reference,” meaning the 
mathematics was mentioned more in passing. A recurring 
example of this was, in the context of looking at results 
from a common assessment, teachers would refer to 
prepared reports and not copies of students’ actual work. 
Teachers would share brief observations or anecdotes about 
what item students answered incorrectly and general claims 
about what seemed to be the cause of the error. These 
comments were short and were not responded to directly by 
other teachers. The Elmwood PLC spent less of their time 
on data analysis or on unit and lesson planning although, 
when on those topics, the Elmwood teachers were more 
likely to have “mathematics discussions.” The instances in 
which both groups were having “mathematics discussions” 
were of interest to better understand the nature of teachers’ 
interactions in PLCs around mathematics and the potential 
for those interactions to develop MKT. 
Research Question 2 

As we outline in our Data Analysis section, to pursue our 
second research question about the ways that teachers’ 
collaboration around mathematics fosters or constrains 
opportunities to develop MKT, we started by taking the 
intervals coded as “mathematics discussions” and 
combined them with sequential intervals to form instances. 
We observed seven instances of mathematics discussion in 
the Brantley team, although only one of those instances 
included multiple intervals. The Elmwood team had six 
instances of mathematics discussion across their four 
observed meetings, with two instances spanning multiple 
intervals. Across the two teams, the three “extended” 
instances of mathematics discussion all occurred during a 
focus on unit and lesson planning. In the sections that 
follow, we look more closely at two of these extended 
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Figure 2. The “box” and “FOIL” methods for multiplying binomials. 


instances—one from each group as they serve as the 
opportunities best suited for teachers to develop MKT due 
to the length and depth of the discussions. However, as we 
will highlight, even these opportunities were ultimately 
limited in their potential impact. 

Brantley PLC: the multiplicative inverse and solving 
linear equations. In this excerpt, all five teachers of the 
Brantley PLC discussed a consideration brought forward by 
Vincent about solving linear equations, which was the 
focus of an upcoming unit. Prior to this excerpt, Kate had 
made a comment about “box” and “FOIL” processes for 
multiplying binomials, as are each illustrated in Figure 2. 
Kate’s comment prompted Vincent to raise a concern about 
focusing on “shortcuts” versus “concepts”: 


This last comment by Luke was not taken up directly and 
was, instead, immediately followed by a discussion about 
the students in the teachers’ classes and issues with 
behavior. Within ongoing discussions about practice, a 
teacher’s distinction between a “concept” and a “shortcut” 
gave rise to an opportunity for the group to consider a 
mathematical topic and the key mathematical ideas 
underlying that topic. Within Vincent’s comments is the 
implicit assertion that the “concept” when solving linear 
equations is that multiplying a coefficient by its 
multiplicative inverse will “undo” that coefficient because 
of the multiplicative identity, and that same multiplicative 
inverse must be multiplied to the expression on the other 
side of the equation to maintain equality. While dividing 
both sides of an equation by the coefficient one is 
attempting to deal with yields a correct answer, Vincent 
contends that it is merely a shortcut (and, as he raises, it is a 
shortcut that is useful or efficient only for certain cases). 

These comments, in the context of working with students 
who are struggling on assessments and an upcoming unit 
on solving linear equations, provided the Brantley PLC 
teachers an opportunity to unpack aspects of these 
mathematical ideas in service of some of the mathematical 
demands of their work in the classroom. Specifically, this 
instance provided the Brantley teachers with an opportunity 
to discuss a range of ways to find a solution to a linear 


I like to teach, you know, if you don’t have a coefficient with the variable you just add the opposite of the 


number to each side; if you’ve got a variable you just multiply both sides by the reciprocal of that coefficient. 


I teach them to undo it. So they don’t recognize that they’re using the inverse operation. They can tell me what 


I want them to get it. I don’t want them to know ‘multiply by the reciprocal’. I want them to solve an equation. 


number is with x and today it gets worse because you have to pick which number you do first. If it’s plus you 
do the opposite. So how do you get rid of a plus? You minus. How do you get rid of a minus? You plus. How 


Vincent: 
In middle school they taught them, well you divide ... 

Kate: See, I divide. 

Vincent: But that’s not the concept in my opinion. That’s just a shortcut. 

Kate: 
the inverse operation is but they don’t necessarily know that that’s what it’s called. Like, ‘How do you undo 
plus two x?’ ... ‘Subtract it.” ‘How do you undo multiplication?’ ... “You divide.’ Because dividing and 
multiplying by the reciprocal are going to end up the same. It’s how you choose to teach it. 

Luke: I disagree. 

Vincent: The problem is when you have like two-thirds x. 

Hillary: Plus they cannot remember how to divide the fractions. I’ve said it I don’t know how many times in class 
because it comes up, like, ‘When you divide, you multiply by the .. .? 

Vincent: The reciprocal. 

Hillary: [laughs] And it’s crickets. 

Vincent: That’s why I try to teach it that way so they’ve got to learn one thing: You either add the opposite or you 
multiply by the reciprocal. That’s all you’ve got to know. 

Kate: 

Samantha: I do it the way [Kate] does it. Yesterday I really hammered home, you know, we’re going to move whatever 
do you get rid of a negative? Because that’s somehow different. . . 

Kate: 


Especially with the number of kids who are in these reading classes, if I put the word ‘reciprocal’ on the board, 


they can’t read it. They understand divide. 
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I’ve been not fighting that battle. What I’ve said is, ‘Now I multiply by the reciprocal. Watch this.’ And we’ve 


done several of them both ways. Then whatever kid I call on, that’s the way we do it. We either divide or we 
multiply by the reciprocal because it’s the same thing. So why care, as long as they get it right? But what 
they’re not getting right is to remember to put the fraction in parentheses when you divide, because they have 
calculators. So as long as they can input their calculator correctly and as long as they are undoing then they are 
doing the correct concept. They are just not doing it in the most, what I would say, efficient way. It’s much 


Another reason that I did it that way—I can’t undo what all their teachers have told them in seventh and eighth 
My problem is that, in Algebra II, you can’t divide matrices. You have to multiply by the multiplicative 


Yeah, but multiplicative inverse and reciprocal . . . like, I know it’s the same but at the same time, like, this is 


the same thing when [Kate] says to these kids, ‘Remember ‘FOIL’ and ‘box.’’ They don’t remember 
multiplying. They remember a process, they remember a catch phrase, they remember a .. . whatever. That’s 
why they remember ‘FOIL’ and ‘box’ because it’s a catch phrase and they know the process. As long as you 
can jar them, they realize they’re multiplying it just takes them a minute or two. ... When they get to matrix, 
these kids, are you telling me that because I did divide by a fraction, they’re not going to get matrix 


Samantha: 
more efficient to multiply by the reciprocal. 
Kate: 
grade. I have to kind of pick up where they left off. 
Luke: 
inverse. 
Samantha: 
multiplicative inverse? 
Luke: 


No but it’s the converse, which is that if they learn to multiply by the multiplicative inverse then when they get 


to matrices, ‘Oh we just multiply by the multiplicative inverse.’ 


(Brantley PLC, Meeting 4, 28:41-35:49) 


equation, the underlying ideas that make the strategies 
work, and the limitations of the strategies. Teachers also 
need an understanding of how concepts build throughout 
the progression of school mathematics, such as how work 
with students on solving linear equations builds on 
students’ prior knowledge and can be constructed in ways 
to be built upon in the future. Luke’s comment about the 
applicability of strategies used to solve linear equations 
when solving equations with matrices brings to light this 
aspect of the work of mathematics teaching. 

However, this excerpt illustrates ways that opportunities 
to discuss mathematics content were pushed aside. While 
Vincent brought up a concern rooted in the mathematics at 
the focus of the discussion and Luke affirmed that concern, 
the discussion of the group relied more on individuals’ 
preferences for presenting the topic—a preference 
determined by a teacher’s perceived individualistic style 
and personal experience instead of the mathematics at hand 
or a greater sense of teaching and learning mathematics. 
Throughout the discussion, any potential press on a 
teachers’ mathematical understanding was deflected to 


Doug: 


issues with students such as understanding vocabulary and 
in-class behavior. As a result, teachers’ own understanding 
of the concepts and strategies discussed could have 
remained unchanged and, thus, not impactful on the way 
they know mathematics for their work as teachers. Even 
when mathematics was specifically discussed, once there 
was an implicit or explicit acknowledgment that all of 
members have a sense of the mathematics or an answer, the 
talk about mathematics became vague or moved away from 
the mathematics altogether. 

Elmwood PLC: doing, undoing, and visualizing 
solving linear equations. This second excerpt also 
focuses on solving linear equations, this time in a 
discussion in the Elmwood PLC. In the excerpt, Doug 
demonstrated a visual model he called the “stacking boxes” 
method (see Figure 3). This occurred during a point early in 
their first meeting when teachers were sharing materials and 
ideas on the unit on solving linear equations. After Jackie 
prompted the group for any other ideas to share after 
initially sharing something from her own practice, Doug 
offered to discuss a strategy he uses with students: 


[After writing 2x + 3 = 10 on paper at center of table] Now you start off with x because it’s the only 


important thing on the page. I keep trying to stress it. I just don’t know whether it’s getting through or 
not, but they keep looking at the ten and the numbers and, ‘What am I supposed to do with the two, the 
three, and the ten?’ You know so, ‘I’ll add a ten to the three and divide by the two and that sounds good 
to me.” They just have no idea so, you know, what I try and do is ... they know order of operations. 


Mine do. 


Sabrina: Mine do, they just can’t do it right. 


122 


(Continued) 


Volume 117 (3-4) 


Math Teacher Development in PLC 


Doug: But they can tell you which one comes first. 

Sabrina: Oh yeah. 

Doug: So in here I’ve got addition and multiplication. The thing that gets done to the x first is multiplying by 
two [expanding on drawn image]. Then the last thing that gets done is adding by three and that equals 
ten. You do all of this. So it’s the algebraic thinking of doing and undoing. 

Pam: Take the boxes down. 

Doug: Now we’ve got to unstack the boxes and when you unstack the boxes you take off the top first. So it’s 
done in reverse, the order of operations, of course. So to take this box off you do ‘minus three’ 
[expanding on drawn image]. And that takes that box off and gives you seven. Then you’ve got to take 
off the multiply by two. How do you take off multiply? 

Jackie: Divide! 

Doug: Right. 

Sabrina: Nice role playing, [Jackie]! 

Doug: [Finishing drawn image] And you get that. 

Sabrina: That’s interesting. I’ve never seen anything like that before. 

Jackie: He made it up! 

Pam: You should get a Ph.D. ... 

Doug: [laughs] Yeah, there’s an idea. 

Jackie: I like visual stuff. 

Doug: I try and push that now. 

Sabrina: Do you do it for one-step, too? Or just two-step? 

Doug: I do it with everything. As a matter of fact, you can do it, somewhat, with multiple steps. Only somewhat. 
If you’ve got on the same side . .. if the x’s are still on the same side, then there’s ways to... 

Jackie: So you haven’t developed anything for a variable on both sides? 

Doug: TE MRGTC 1530-1 oe 

Jackie: I’m guessing there is. You can do it. You need to think about that. [laughs] 

Doug: Okay. Next week I’1l come with something on that. 

Jackie: [after a pause in discussion] Other common unit stuff? 


(Elmwood PLC, Meeting 1, 5:35-9:10) 


Also in the context of collectively discussing an 
upcoming unit, this excerpt shows another type of 
opportunity for teachers to meaningfully discuss 
mathematics in PLCs. Doug brought forward a method that 
he used with his students for solving particular forms of 
linear equations. It is important to acknowledge that Doug’s 
method was a visual representation of a solution strategy 
and, as a result, was not intuitive. The representation 
corresponds to the steps one uses when solving a linear 
equation through symbolic manipulation. The discussion 
around Doug’s strategy provided an opportunity for 
teachers to unpack the concepts that underlie procedures 
used to solve linear equations and, thus, potentially develop 
MKT. 

In presenting his idea, Doug modeled the strategy as a 
“teacher” with the others observing and interacting as 
“students,” brought to the forefront with Jackie’s response 
(“Divide!”) to Doug’s leading question (“How do you take 
off multiply?) and Sabrina’s acknowledgment of Jackie’s 
“role playing.” Doug created a written representation of the 
work involved with the method on the particular problem 
(2x +3= 10), which provided a concrete record for 
teachers to discuss. The conversation at the end of the 


School Science and Mathematics 


Elmwood vignette is noteworthy as Doug was pressed to 
consider the limitations of the method. While this press was 
responded to with uncertainty from Doug and a claim that 
lacked any backing from Jackie (“I’m guessing there is’’), 
overall the discussion took a different form than that of the 
Brantley teachers. Instead of evaluating an idea from the 
standpoint of how that idea aligned with one’s personal 
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Figure 3. Doug’s 


stacking boxes” method of solving linear equations. 
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style, the Elmwood PLC talked about Doug’s method in a 
way that at least opened up the opportunity for them to 
think more about the method and the underlying 
mathematics (although many comments and mathematical 
ideas remained unchecked). 


Developing MKT in PLCs: Discussion and 
Implications 

Our analyses provided insight into the interactions of two 
teams of mathematics teachers—each operating under the 
premise of being a PLC—and the way in which their 
collaborative interactions provided space for teachers to 
develop MKT. Both groups had positive collaborative 
norms, with each group having some general foci (such as 
looking at assessment data or engaging in common 
planning) and engaging in what might be characterized as 
dynamic discussion (involving all teachers, turn-taking, 
sharing of tentative ideas). Yet, core to our study, 
discussions specifically focused on mathematics in these 
groups were rare. Additionally, even in those few 
opportunities, there were many factors at play that hindered 
the potential development of MKT. In this section, we 
highlight two considerations that arise from these findings 
related to considering the role PLCs, in general, can play in 
teachers’ mathematical development. 

Multiple Goals of PLC Interactions and the Role of 
Teachers’ Mathematical Development 

In addressing our first research question regarding the 
extent to which these teachers’ discussions focused on 
mathematics content in their PLCs, we found that teachers’ 
discussions seldom included reference to (and far less a 
focus on) specific mathematics content. However, our 
findings also highlight multiple goals that can be driving 
the substance and form of teachers’ interactions in PLCs. 
These goals may be prescribed by a district or school or 
may originate from teachers’ own needs. Even in a group 
comprised entirely of mathematics teachers, those goals 
might not always be taken up in ways that are specifically 
mathematical and, furthermore, might not involve teachers’ 
own mathematical development. 

This raises the question of why we would expect to see 
more in terms of mathematics discussions and opportunities 
for teachers’ development of MKT if that was not a purpose 
of these teachers’ interactions. With this work, we address an 
assumption that mathematics teacher collaboration around 
their mathematics teaching practice would be inherently 
mathematical. Our findings show that is not reliably the case. 
A similar question to ask is whether or not the lack of 
mathematics discussions among these teachers was a sign of 
them being unproductive, even if they were attending to 
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other stated goals. We do not see this work as a claim that 
evaluating the whole of these teachers’ interactions should 
be based only on their mathematical focus, as other 
investigations could focus on teachers’ use of assessment 
data, student work, or common planning. However, our 
findings suggest that the way PLCs have been broadly 
characterized—as a panacea for all things related to student, 
teacher, and school improvement, including teachers’ 
content knowledge—is an overstatement. This is not a 
wholesale critique of PLCs, although it is an assertion that 
expectations for PLCs involving teacher development, 
specifically around content knowledge, will need to be 
supported in specialized ways, such as the case with work 
described by Borko and her colleagues (2015) and others. 
Our findings raise doubts about the ways that a general 
notion of teacher collaboration, even among mathematics 
teachers exclusively, will necessarily support teachers’ 
mathematical development. 

The Specialized Nature of Doing and Discussing 
Mathematics as Teachers 

From our work toward our second research question, we 
also highlight how the lack of opportunity for teachers to 
develop MKT in PLCs is not just a matter of time being 
spent on other things. Even if teachers were to spend more 
time discussing specific mathematics content, the vignettes 
we shared demonstrate some of the ways in which those 
conversations would continue to be limited in terms of their 
mathematical focus. Based on our findings and on previous 
research on teacher PD, we highlight two additional factors 
that play a role in the nature of teachers’ collaborative 
mathematics discussions. 

First is the way in which teachers’ conversations about 
mathematics need to be different that mathematical 
discussion with students because of differences in the 
purposes of such discussions (Elliott et al., 2009). In our data 
we saw two ways in which mathematics discussions played 
out—both with shortcomings when it comes to the potential 
for teacher development. The example from the team at 
Elmwood showed teachers engaging with a procedure 
brought forth by Doug in a way that was reflective of how 
students might engage with the teacher and the content. 
While teachers learned about Doug’s idea, that alone did not 
engage teachers with the mathematics underlying that idea. 
This form of “modeling” of instruction is not sufficient for 
engaging teachers in mathematics. 

With the Brantley team, we saw mathematical 
conversation that did not look like what you might expect 
to see in a school classroom, although the discussion was 
also not rich in terms of the opportunities for teachers to 
develop MKT. In the case of the Brantley team, there was 
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most often a distance from the mathematics, even when 
mathematics was being referenced or discussed. The 
teachers only interacted verbally, with no ideas being 
recorded or built upon. The discussion also repeatedly 
moved to non-mathematical topics, such as_ student 
behavior or school issues. The segments of conversation 
that were focused on mathematics were comprised mostly 
of broad statements about mathematics and statements of 
teachers’ personal preferences, as opposed to about the 
substance of the mathematics at hand. We found it 
interesting that a group of mathematics teachers 
collaborating around their mathematics teaching practice 
would not be sufficient for creating discussions that were 
mathematical in nature. However, these practices are 
reflective of issues reported by others, such as Elliott and 
her colleagues (2009), regarding teachers’ reluctance to dig 
into mathematics with peers in PD settings because of 
status issues or assumptions that are made. 

One way to support teachers’ discussions of 
mathematics would be the role of a skilled facilitator 
(Borko et al., 2015; Elliott et al., 2009; Fulton et al., 
2010; Lesseig et al., 2016). In our data, such a skilled 
facilitator was not present. Teachers from within the 
group took on leadership roles—sometimes with 
multiple people taking on different aspects of leadership 
needs. Those needs included maintaining an agenda for 
group meetings and attending district-wide meetings. 
Other leadership roles, such as department chair, could 
have also had an influence on the work of the group. 
However, none of the teachers on either team were 
tasked with or prepared for facilitating teachers’ 
engagement with mathematics. With the presence of such 
a skilled facilitator and an explicit focus on developing 
MKT, examples such as those put forward by Borko and 
her colleagues (2015) and Brodie (2014) show that 
teachers can work meaningfully on mathematics in the 
context of their collaborations. However, our data show 
that without such support, the likelihood of teachers 
developing these mathematical resqurces through their 
practice-focused conversations is greatly diminished. 


Conclusion 

In this article, we examined the extent to which 
mathematics teachers collaborating as a PLC focused on 
mathematics content as part of their discussions and how 
their discussions about mathematics might have provided 
opportunities for teachers to develop MKT. By focusing on 
MKT, we echo the recent-work of many who are specifying 
the nature of teachers’ mathematical knowledge and the 
ways in which such knowledge is developed (Ball et al., 
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2008; Borko et al., 2015; Elliott et al., 2009). We found that 
even with groups of mathematics teachers talking within 
the context of their mathematics teaching, mention or 
discussion of specific mathematics content was rare. In the 
event of conversation that did focus on _ specific 
mathematics, the prevailing ways of working in the group 
hindered the potential of those discussions as opportunities 
for teachers to develop MKT. We see these findings 
contributing to a needed discussion around whether PLCs, 
especially those not formed as part of a more specified PD 
effort or provided other specialized supports, can reliably 
and consistently serve as a site for teachers’ mathematical 
development. These findings do not cast doubt on the 
potential of benefits of PLCs in general, nor do they negate 
the impact of the small set of examples in the literature of 
successful PLCs focused specifically on teachers’ 
mathematical development. Given the way research and 
anecdotes on school improvement are taken up at school, 
district, and state levels, these findings present the need for 
future work on PLCs (and PD in general) to take more care 
in reporting on the way a design is intended to support 
teachers and its ultimate impact. 
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This study examined primary grades students’ achievement on number sense tasks administered through an Internet- 
based formative assessment tool, Assessing Math Concepts Anywhere. Data were analyzed from 2,357 students in 
teachers’ classrooms who had participated in a year-long professional development program on mathematics formative 
assessment, 1,427 students from teachers who had participated in the program in the year prior, and 9,783 students 
whose teachers had not participated at all. Analyses indicated that all students in the treatment group demonstrated 
growth, and that student achievement was influenced by the number of times the assessment was used to collect data and 
make instructional decisions. Further, there was a relationship between districts’ socioeconomic status and growth, 
meaning students from impoverished backgrounds grew more than their peers. 


Overview 

The Need to Improve Students’ Mathematics 
Achievement 

In the current era of educational accountability, educational 
leaders and researchers are constantly seeking ways to 
increase students’ achievement and learning. While a 
teachers’ effectiveness continues to be identified as the most 
influential variable on students’ learning, there is a need to 
more explicitly identify the teacher practices and 
characteristics that have the greatest benefit to students 
(Darling-Hammond, Wei, Andree, Richardson, & Orphanos, 
2009; Hattie, 2009; Nye, Konstantopoulos, & Hedges, 2004). 

Research studies have cited efforts to positively impact 
student achievement specifically chronically low-performing 
students, especially those from impoverished backgrounds 
(National Center for Educational Statistics [NCES], 2013; 
Organisation for Economic Co-operation and Development, 
2012). Analyses of data collected from the Programme for 
International Student Assessment indicate vast differences 
across the world about how mathematics is taught as well as 
varied levels of performance on the assessment. These 
differences, while somewhat associated to levels of poverty, 
are largely caused by differences in teacher quality and the 
types of pedagogies used during mathematics instruction 
(Hattie, 2011; Nye et al., 2004). 
Targeted Teaching and Formative Assessment 

Benefits of formative assessment. Worldwide, the idea 
of using data to inform teachers’ instruction has been 
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advanced (Goss, Hunter, Romanes, & Parsonage, 2015; 
Means, Padilla, DeBarger, & Bakia, 2009). Goss et al. 
(2015) cited the positive impact of targeted teaching, in 
which teachers design instruction that is differentiated and 
designed to target students’ individual needs. More than 
ever, classrooms have a wide variety of ability levels in a 
classroom (Santiago, Donaldson, Herman, & Shewbridge, 
2011), and studies indicate that streaming curriculum by 
teaching without differentiation or targeted instruction has 
a negative impact on student achievement (Education 
Endowment Foundation, 2015). Black and Wiliam (1998) 
also found that using data to make instructional decisions led 
to significant gains in student achievement in mathematics. 

In a meta-analysis of over 800 studies Hattie (2009, 
2011) identified specific educational factors that are 
significantly associated to positive student achievement. 
One of the most impactful factors is teachers’ use of 
formative assessment strategies. Multiple meta-analyses of 
research studies have found that, when implemented well, 
formative assessment practices can effectively double the 
speed of student learning (Wiliam, 2007/2008). 

Formative assessment practices involve the process 
of collecting student data, analyzing it, and using it to 
plan, modify, and implement instruction that addresses 
students’ specific needs. In this article, we refer to 
formative assessment to the entire process of all of these 
activities. Empirical studies show the benefit of formative 
assessment practices on student achievement in literacy 


127 


Influence of an Internet-Based Formative Assessment Tool 


(Fuchs & Fuchs, 1986) as well as mathematics (Wiliam, 
2007, 2010). Formative assessment refers to the evaluation 
of student learning during the learning process with student 
learning as its main priority. The literature suggests several 
components of effective formative assessments: (a) 
opportunities for teachers and students to express learning 
goals; (b) use feedback to move forward; (c) engage in peer 
assessment; (d) uncover student reasoning; and (e) create 
follow-up instruction that extends thinking (Baroudi, 2007; 
Heritage, 2007; Hodgen, 2007; Huinker & Freckmann, 
2009). Empirical studies indicate that student achievement 
significantly increases when teachers use formative 
assessments appropriately (Black & Wiliam, 1998; Wiliam, 
2007). Teachers who use formative assessment are able to 
better understand the learner and use the data for the 
development of individualized instruction for each specific 
learner. 

A meta-analysis was employed to examine the effects of 
systematic formative evaluation and noted that the effect 
sizes were significant for students whose learning processes 
were monitored systematically (Fuchs & Fuchs, 1986). 
Moreover, effect sizes were even higher when teachers 
used data-utilization rules. Fuchs and Fuchs (1986) work 
also found that for struggling learners who are sometimes 
referred to as at-risk, formative assessment practices started 
to close the achievement gap in literacy. Related studies are 
needed in the field of mathematics. 

Internet-based tools for formative assessment. In the 
21st century, there have been many formative assessment 
tools or programs developed and used in order to support 
teachers’ planning and implementation of differentiated 
instruction. Some of these are used exclusively for the 
collection and analysis of student performance only, 
requiring teachers to analyze data and plan further 
instruction. Others are generative or adaptive learning 
programs that include opportunities for teachers to 
formatively assess their students as well as instructional 
activities that are completed on an Internet-connected 
device such as a computer or iPad. 

The commercial companies that produce the generative 
and adaptive programs have data that shows that the use of 
their program leads to gains in student learning on either 
their assessment items or in some cases state-wide 
assessments (Center for Educational Policy Research, 
2016; Curriculum Associates, 2016; Educational Research 
Institute of America, 2015). However, these evaluation 
studies indicate that not every student benefits from the use 
of these tools, control groups are not used to compare 
students who use these programs with those who do not, 
and there are other confounding variables not controlled for 


128 


such as other mathematics resources and the quality of 
mathematics instruction or teacher characteristics. Still, 
these evaluation reports indicate a need to further investigate 
how these programs are used and how they influence 
student learning. 

The other programs, such as the one examined in this 
study, provide teachers with formative assessment data on 
their students’ performance that teachers can then use to 
plan subsequent instruction to meet their students’ learning 
needs. In a recent study, Herndon (2015) found that 
teachers were using MClass, an Internet-based formative 
assessment tool for reading, but struggled to use the data to 
plan instruction that specifically aligned to their students’ 
reading levels. 

In mathematics, the only research published on teachers’ 
use of formative assessment data collected through an 
Internet-based tool has been done by the authors of this 
current study. Past studies echo the findings of the 
Herndon (2015) study in that teachers assessed their 
students using AMC Anywhere, but reported difficulty 
finding the time and knowing exactly how to reconcile 
their assessment data, their mathematics resources, and 
the state mathematics standards (Martin, Polly, Wang, 
Lambert, & Pugalee, 2016). Still, some studies from 
AMC Anywhere document evidence that despite 
reporting struggles that teachers successfully analyzed 
data, designed and implemented instructional activities 
that aligned to their students’ mathematics performance 
(Polly, Martin, Wang, Lambert, & Pugalee, 2016). 
Formative Assessment in Mathematics 

In mathematics education, formative assessment has long 
been identified as a high-leverage and impactful instructional 
practice (NCTM, 2000, 2015; Wiliam, 2007; Wiliam & 
Thompson, 2007). Formative assessment research cites that 
the practice is most valuable to the teaching and learning 
when teachers use data to modify instructional goals, 
instructional activities, and instructional pedagogies 
(Heritage, 2007). 

The research base has documented teachers’ difficulties 
carrying out the various aspects of formative assessment 
practices (Pfannkuch, 2001; Polly et al., 2014; Wiliam & 
Thompson, 2007). Abrams (2007) noted that teachers opted 
to not collect data frequently due to the time demands 
associated with planning and teaching their standards. In 
essence, teachers did not feel that they had the time to 
implement formative assessment. In New Zealand, 
Pfannkuch (2001) conducted a 10-year longitudinal study 
and found that while primary-grade teachers used a variety 
of assessment practices, they were focused on mastery 
learning and did not include a lot of formative or informal 
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assessments of students’ mathematics learning. She found 
an abundant number of assessments meant to assess 
mastery at the end of teaching rather than formative 
assessment practices during instruction. 

Higgins and Parsons (2009) researched the nationwide 
professional development project in New Zealand and 
found that teachers were able to effectively support their 
students’ development of number sense through an 
integrated professional development model that included a 
research-based framework about number sense, the use of 
diagnostic interviews with students, and a strategy-specific 
teaching model. Through classroom observations and 
teacher interviews, New Zealand teachers shared about 
their increased use of diagnostic interviews and formative 
assessment practices as well as the gains in their students’ 
understanding of number sense (Higgins & Parsons, 2009). 
Formative Assessment on Children’s Number Sense 

Research studies from the same project as this present 
study indicate that professional development focused 
on diagnostic interviews and the process of making 
instructional decisions based on the data was associated with 
students’ achievement in number sense (Martin et al., 2016; 
Polly, Martin, Wang, Lambert, Pugalee, & Middleton, 
2016). In terms of number sense, we refer to what Gersten 
and Chard (1999, p. 18) referred to as “‘a child’s fluidity and 
flexibility with numbers, the sense of what numbers mean 
and an ability to perform mental mathematics and to look at 
the world and make comparisons.” 

Within the five aspects of mathematical proficiency 
described by National Research Council (2001), this study 
focused on students’ conceptual understanding and procedural 
fluency as it relates to students’ number sense skills. 
Conceptual understanding is defined as the comprehension of 
mathematical concepts, operations, and relations and whether 
or not students are able to demonstrate their knowledge 
through the creation and manipulation of mathematical 
representations. Meanwhile, procedural fluency refers to 
skills in carrying out procedures flexibly, accurately, 
efficiently, and appropriately, such as the mathematical 
operations. In primary grades mathematics, the 
foundational number sense skills focus on counting, 
addition and subtraction fluency, and building an 
understanding of ten and place value (Richardson, 2012). 

Clearly, formative assessment practices and teaching that 
is driven by data and targeted to meet students’ needs has 
potential to increase students’ achievement in mathematics. 
However, there is a need to examine how to best support 
teachers’ use of formative assessment practices as well as a 
need for more research about the influence of formative 
assessment on students’ achievement in mathematics. This 
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study focuses on teachers’ use of formative assessment with 
primary-grade students about their understanding of 
number sense. 
Purpose and Research Question 

This study focuses on the research question: Is there a 
significant relationship between teachers’ use of the Internet- 
based formative assessment tool AMC Anywhere and 
student growth in mathematical skills? Specifically, variables 
such as poverty, school size, and the frequency of the use of 
formative assessment practices were examined as possible 
influences on the variance of students’ achievement. 


Methods 
Context of the Study 

The data analyzed in this study came from teacher- 
participants in a year-long professional development project 
funded by the Mathematics Science Partnership grant 
program in the United States. Teachers participated for one 
year in 80 hours of learning about formative assessment, 
how children develop number sense, and the use of an 
Internet-based formative assessment tool, Assessing Math 
Concepts (AMC Anywhere, Richardson, 2012). Teachers 
also learned about using the data from the tool to select and 
modify instructional activities from the Developing Number 
Concepts (DNC) curricular resources (Richardson, 1998). 
DNC resources were only available to those in the 
professional development and participants from the 
previous year. All participating teachers, including 
comparison group teachers, were also using the reform- 
based mathematics curricular resources, Investigations in 
Number, Data, and Space (TERC, 2007). Therefore, the 
project also included support aligning the concepts in the 
assessment to curricular resources and the state 
mathematics standards, which were the Common Core 
State Standards for Mathematics (CCSSI, 2011). 

AMC Anywhere is an Internet-based tool that supports 
the process of conducting diagnostic interviews on 
students’ mathematics understanding. Teachers use AMC 
Anywhere in a one-on-one setting, in which students solve 
tasks either with manipulatives, such as counters, 10 frame 
mats, or snap cubes, or using mental mathematical 
reasoning. Teachers pose tasks to students, who complete 
the tasks, and then the teacher records students’ responses 
and strategies in the tool. The tool includes a built-in rubric, 
which then provides teachers with a rating on the 
assessment, which aligns to the instructional materials from 
the DNC resources. The expectation in this project is that 
teachers used the assessment with every student in their 
class and then used the data to plan and implement 
differentiated, targeted instruction. 
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Figure 1. Screenshot of Hiding Objects Recording Screen. [Color figure can 
be viewed at wileyonlinelibrary.com] 


While AMC Anywhere includes nine different 
assessments, this study examined only student learning data 
from the Hiding Assessment. The concepts in that 
assessment align to decomposing a number into two 
smaller numbers in kindergarten, missing addend and 
decomposing a number in Grade 1, and addition and 
subtraction fluency in Grade 2. 

The Hiding Assessment includes two separate parts with 
distinct tasks. Part One requires students to find the missing 
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Figure 2. Screenshot of Classroom Instruction Report for Hiding 
Assessment. [Color figure can be viewed at wileyonlinelibrary.com] 
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addend of numbers using counters to assist students. For 
example, in Figure 1 students count out 10 counters and the 
teacher then removes 8 counters when the students are not 
looking. The student looks at the pile, sees two counters, 
and has to then determine how many counters the teacher 
took away or is hiding. Part Two consists of the same tasks, 
but without counters and only mental reasoning. Both parts 
assess students’ ability to differentiate ways to solve tasks 
involving the numbers 3 through 10. 

As an Internet-based tool, AMC Anywhere allows 
teachers to collect, store, and analyze data from the 
diagnostic interviews throughout the school year as well as 
across school years for students. Further, its reporting 
features provide teachers with information about students’ 
instructional needs as well as links to the accompanying 
curricular resources, DNC (Richardson, 1998), which 
provide activities for students to explore related to the skills 
in the assessments. The DNC activities can be differentiated 
easily to meet students’ mathematics needs. Figure 2 shows 
a screenshot of the classroom report from AMC Anywhere, 
while Table 1 provides an example about how the 
assessments link to instructional activities. The goal of 
AMC Anywhere and DNC is that teachers would use 
formative assessments as an integral part of the process of 
analyzing students’ mathematical understanding. 
Participants 

Participants in this study include 906 teachers and their 
13,567 students in Grades K—2 in elementary schools in 
North Carolina. Of these 13,567 students, 763 (6%) were in 
kindergarten, 5,815 (43%) were in the first grade, and 6,989 
(51%) were in the second grade. These students were put 
into three groups: treatment group (n = 2357, 17%), prior- 
year group (n= 1427, 11%), and comparison group 
(n = 9783, 72%), Treatment group students are from the 
127 (14%) teachers who participated in the 80-hour 
professional development who were asked to use AMC 
Anywhere at least three times as part of the project. Prior- 
year group students are from the 81 (9%) teachers who 
participated in the professional development program last 
year who still had access to AMC Anywhere and were 
encouraged by their school district to still use the 
assessment. The rest of the teachers (n = 698, 77%) and 
their students served as the comparison group. The teachers 
in the comparison group used AMC Anywhere, but never 
participated in any professional development about using 
the data as part of the formative assessment process. The 
data from all the students were available to the researchers 
through the AMC Anywhere website. These data are 
anonymous, and we have obtained the waiver of consent 
forms from these participants with the approval of the 
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Table 1 
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Chart Aligning the Hiding Assessment to Instructional Activities (Used with permission from Richardson, 2012) 


Independent Activities for Needs Instruction (I) To6 To 10 
2:2-14 Number Arrangements: Using Cubes Xx Xx 
2:2-15 Number Arrangements: Using Color Tiles Xx x 
2:2-16 Number Arrangements: Using Toothpicks Xx x 
2:2-17 Number Arrangements: Using Collections X x 
2:2-18 Counting Boards: Making Up Number-Combinations Stories X xX 
2:2-20 Number Shapes: Using Number Cubes Xx x 
2:2-21 Number Shapes: Using Spinners x X 





Institutional Review Board at our university. Since this is 
not an experimental design, we are aware of the potential 
differences in teacher and student characteristics of the 
three groups of teachers with their students. These 
differences make simple mean comparisons across the 
groups meaningless. Therefore, we adopted a linear growth 
model that estimates the initial status of the student’s 
mathematics achievement at the beginning of the treatment 
and the monthly growth rate for all students. Linear 
growth models accommodate various initial status as well 
as nonsimultaneous data collection at the same time 
(Raudenbush & Bryk, 2002). 

All teacher-participants hold a state license in 
elementary education which spans from kindergarten 
through Grade 6 (ages 5 through 11). Additionally, all 
teachers in the study use an Internet-based formative 
assessment tool, MClass (Amplify, 2015), which the 
state mandated to assess students’ literacy development. 
The state has not mandated the use of a formative 
assessment tool for mathematics. 

Data Analytical Procedure 

The researchers used Item Response Theory to transform 
the teachers’ assessment of student mathematical skills 
from the original letter grade scaling into interval-level 
scale scores using the Rasch model (Andrich, 1978; Martin, 
Lambert, Wang, & Polly, 2017) with Winsteps software 
(Linacre, 2012). The standardized scaled scores have a 
mean of 500 and a standard deviation of 100. Since 
students were assessed multiple times during the academic 


year, month was used as the unit of analysis in time so that 
each assessment is associated with the number of months 
elapsed from the start of the academic school year (August 
15). Due to the nature of nested data in our project (student 
data are nested within classrooms and classroom data are 
nested within schools), a multilevel data analytic procedure 
(i.e., hierarchical linear models) was adopted to account for 
the variances between classrooms as well as between 
schools (Raudenbush & Bryk, 2002). A_ four-level 
hierarchical linear model (HLM4) was employed to predict 
the student initial status of mathematical skills as well as 
the monthly growth rate for the students during the 
academic year at Level One. Level One is simply a 
linear growth model where the dependent variable is the 
student performance on the mathematics achievement 
measure (i.e., hiding task) and the independent variable 
is time (coded as 0 if the measurement took place in 
August, 1 in September, 2 in October, etc. so that the 
slope of the regression represents monthly increase). 
The number of assessments each student received 
(Assessment) was used as a predictor (group-mean 
centered) at Level Two (Student Level). At Level Three, 
the average number of assessments in each classroom 
(Mean_Assess) and the group status (treatment group 
versus comparison group) were entered as predictors 
[group-mean centered, Classroom Level]). Finally, the 
school size (size) and the percentage of free/reduced 
price lunch children (poverty) were used as predictors 
(grand-mean centered) at Level Four (School Level). 


Table 2 
Descriptive Statistics for Student Performance on Hiding Assessment 
Part One Part One Part Two Part Two 
(Initial) (Final) (Initial) (Final) 
Prior-Y ear 447.55 (66.96) 587.81 (76.17) 458.32 (65.72) 586.63 (70.58) 
Treatment 439.59 (60.07) 570.88 (65.52) 450.68 (61.25) 571.49 (67.35) 
Comparison 441.16 (70.26) 560.61 (71.87) 453.51 (68.23) 559.74 (72.61) 





Note. Numbers in parentheses are standard deviations. 
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Table 3 
Estimation of Fixed Effects for HLM4 of Hiding Assessment 


Saar ee ee ee ee eee ee a a a 





Part One Part Two 

Level Estimate SE df Estimate SE df 

Intercept 1 411.62 1300747 135 453.87 13.1853" 150 
Assessment 2 =—1'().03 DL Ee™ Zoi —9.64 3.34** 2340 

Mean_Assess 3 4.36 4.66 34 0.87 6.77 37 

Group 3 — 32,07 10:252* 34 —34.03 Tract” oy 

Size 4 —.01 02 135 =.05 O22 * 150 

Poverty 4 —1.94 ee 135 —1.42 02305" 150 
Slope 1 11.58 L277 e™ 2912 7.30 1.46*** 1664 
Assessment 2 45 So 2912 97 42" 1664 
Mean_Assess 3 78 30** 2912 .62 47 1664 
Group 3 .90 62 2912 21 he 1664 
Size 4 .001 .002 2912 .008 002 a 1664 
Poverty 4 05 .02** 2912 01 02 1664 





Note Ep <,012F"* p< 001, 


Results 
Group Mean Differences 

Descriptive statistics of student performance on the 
Hiding assessment are reported in Table 2. Only the initial 
status and the final score of the students in each of the three 
groups were included because each student was measured 
multiple times (ranged from 1 to 13). The assessment that 
was administered at the beginning of the school year (1.e., 
August or September) was treated as the initial status, 
whereas the assessment that was administered at the end of 
the school year (i.e., April or May) was treated as the final 
score of the student performance on the Hiding assessment. 
Estimates of the Initial Status and Monthly Growth 
Rate (Part One) 

The parameter estimates of the fixed effects for HLM4 
are presented in Table 3. In Part One of the Hiding 
Assessment, a comparison group student’s performance in 
an average classroom and at a school of average level of 
poverty is estimated to be 411.62 at the beginning of the 
academic year (August 15), and the monthly growth rate of 
the student is estimated to be 11.58, which means that a 
student’s performance on this assessment is expected to 
increase by 11.58 every month. Students from poverty 
schools were at significantly lower levels at the beginning 
of the academic year, ¢ (135)= —6.24, p<.001. This 
negative relationship between poverty level and student 
achievement is manifested as: With an increase of one 
percent of students who are qualified for free/reduced price 
lunch program in the school, students’ achievement in this 
assessment is estimated to be 1.94 lower. 

Across all the students, treatment group students’ initial 
performance on the assessment was significantly lower than 
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that of the comparison group students, ¢ (34) = —3.13, 
p = .004. Students in the treatment group were expected to 
be 32.07 less than that in the comparison group at the 
beginning of the academic year. This means that the 
teachers who volunteered to be in the comparison group are 
from classrooms whose students are above the average 
level of performance, assuming that all the treatment group 
students are from average classrooms. Moreover, the 
number of assessments that each student received is 
negatively related to the estimated initial status of 
the assessment, ¢ (2557) = —3.71, p<.001. With each 
additional assessment, a student’s initial status is expected 
to be 10.03 less, which suggests that the teachers used 
AMC Anywhere more with the students who had more 
difficulty in learning mathematics. The other two predictors 
of the intercept (classroom mean number of assessments 
and school size) did not have any statistically significant 
impact on the estimation of the initial status of the 
assessment. 

As for the predictors of the slope, classroom mean 
number of assessments had a statistically and significantly 
positive impact on the monthly growth rate, ¢ (2912) = 
2.62, p=.009. For each additional assessment in the 
classroom level, students’ monthly growth rate is expected 
to increase by .78, i.e., from 11.58 to 12.36. Similarly, the 
school average poverty level also had a significantly 
positive impact on the monthly growth rate, ¢ (2912) = 
2.88, p = .004. With an increase of one percent of students 
eligible for the free/reduced price lunch program, the 
students’ monthly growth rate is expected to increase 
by .05, i.e., from 11.58 to 11.63. The other predictors in 
the model (school size, classroom mean number of 
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assessments, and the number of assessments for each 
student) did not have any statistically significant impacts on 
the monthly growth rate. 

Estimates of the Initial Status and Monthly Growth 
Rate (Part Two) 

For Part Two of the Hiding assessment, a comparison 
group student’s performance in an average classroom and at 
a school of average level of poverty is estimated to be 
453.87 at the beginning of the academic year (August 15), 
and the monthly growth rate of the student is estimated to be 
7.30, which means that a student’s performance on this 
assessment is expected to increase by 7.30 every month. 
This monthly growth rate (slope) is statistically significantly 
different from zero, t (1664) = 5.01, p < .001. Students from 
large schools (more than a thousand students) were at 
significantly lower levels at the beginning of the academic 
year in comparison to small schools (less than 500 students), 
t (150) = —2.98, p = .003. Students from poverty schools 
were also at significantly lower levels at the beginning of the 
academic year, t (150) = —4.29, p < .001. With an increase 
of one percent of students who are qualified for free/reduced 
price lunch program in the school, students’ achievement in 
this assessment is estimated to be 1.42 lower. Treatment 
group students’ initial status of the assessment was 
significantly lower than that of the comparison group 
students, ¢ (37) = —3.05, p = .004. Students in the treatment 
group were expected to be 34.03 less than that in the 
comparison group at the beginning of the academic year. 
Moreover, the number of assessments that each student 
received is negatively related to the estimated initial status of 
the assessment, ¢ (2340)= —2.89, p=.004. For each 
additional assessment, a student’s initial status is expected to 
be 9.64 less, which suggests that the teachers targeted at the 
students who had more difficulty in learning mathematics 
and gave these students more assessments during the 
academic year. The other predictor of the intercept 
(classroom mean number of assessments) did not have any 
statistically significant impact on the estimation of the initial 
status of the assessment. 

As for the predictors of the slope, school size had a 
positive impact on the monthly growth rate, t (1664) = 3.54, 
p<.001. With an increase of 100 students in the school 
population, students’ monthly growth rate is expected to 
increase by .20, ie., from 7.30 to 7.50. Similarly, the 
number of assessments each student received also had a 
significantly positive impact on the monthly growth rate, 
t (1664) = 2.88, p = .004. For each additional assessment, 
the student’s monthly growth rate is expected to increase 
by .42, i.e., from 7.30 to 7.72. The other predictors in the 
model (school poverty level, classroom mean number of 
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assessments, and the treatment or comparison group status) 
did not have any statistically significant impacts on the 
monthly growth rate. 


Discussion 

The findings from this study related to the significant 
relationships between school-level poverty, school size, and 
the number of assessments on the monthly growth rate of 
student achievement warrant further discussion. Additionally, 
there are implications for future research and practice based 
on this study. 

Relationship Between Poverty Level and Student 
Achievement 

In Part One of the assessment where students found 
missing parts of numbers using counters, the school-level 
poverty had a positive impact on the growth rate, and 
students from schools with lower socioeconomic status 
showed statistically significant growth compared to others. 
These findings must be taken with caution due to the 
difference in achievement at the beginning of the study. 
Students in poverty and large schools were found to be 
significantly lower-achieving in mathematics at the 
beginning of the school year, which is a possible 
explanation why these students are relatively making a 
faster growth. This finding suggests that the use of the 
AMC Anywhere formative assessment tool led to growth in 
this disadvantaged group of students, and helped to 
decrease the achievement gap between impoverished 
settings and other students. 

The differences in student achievement between students 
from higher and lower socioeconomic settings have been 
noted in large-scale studies (NCES, 2013, 2015). While 
prior studies have identified positive relationships between 
formative assessment and student achievement (Hattie, 
2011; Polly et al., 2014; Wiliam & Thompson, 2007), this 
study found statistically significantly greater gains for 
impoverished students as a result of formative assessment 
in mathematics. This extends earlier work done in literacy 
(Fuchs & Fuchs, 1986), only this study focuses specifically 
on number sense with primary grades’ students. Future 
studies should explicate and further detail the processes of 
teachers enacting formative assessment practices for all 
students within classrooms to see if their decisions and 
actions differ depending on their students’ abilities. 

School Size and Student Achievement 

In Part Two of the assessment, school size had a positive 
impact on the growth rate. Larger schools were associated 
with higher growth rates. This finding, however, does not 
mean that students in larger schools have better performance 
on the assessment because students in larger schools were 
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found to have lower initial status at the beginning of the 
academic year. The higher monthly growth rate can be 
explained by the well-known effect that disadvantaged 
schools show more growth. One possible explanation for 
this higher growth rate in large schools could be the amount 
of support and collaboration at larger schools, which led to a 
higher fidelity of implementation of AMC Anywhere and 
formative assessment processes. In prior work, researchers 
found that teacher collaborative structures, sometimes 
termed professional learning communities, led to greater 
implementation of reform-based practices (Dufour, Dufour, 
& Eaker, 2008; Martin & Polly, 2015). 

In the school districts that participated in the project, 
larger schools often had mathematics-specific support, such 
as a mathematics facilitator or coach. These personnel 
typically assist in instructional planning, and preparation for 
the enactment of specific practices, including formative 
assessment (Martin & Polly, 2015). Future work should see 
what specific aspects these larger schools have that may 
lead to these statistically significant differences. 

Number of Assessments and Student Achievement 

Another noteworthy finding was the positive relationship 
between the number of assessments for students and the 
monthly growth rate of student achievement. The 
implication of these findings suggests that teachers should 
be encouraged to use assessments more frequently in 
classroom instruction not only to better understand the 
student’s status of learning but also to make the students 
aware of their own progress. This interpretation is really 
important, because a close examination of the frequency 
distribution table suggests that most of the students (99%) 
were assessed less than eight times. Students who were 
assessed between 8 and 13 times only consisted of 1% of 
the sample. We understand that it is not only the dosage 
effect but the use of formative assessment that provides 
feedback to the teacher that helps the students improve their 
number sense. The information that the teacher gained from 
analysis of these formative assessments guided their 
instruction and made differentiated instruction possible. A 
close look at the scatter plot (Figure 3) between the number 
of assessment and the student performance in the end of the 
academic year suggests that students who were assessed 
more than three times (which is required for all teachers 
who participated in the professional development program) 
were all above the average. While this work is similar to 
prior findings (Polly et al., 2014; Wiliam & Thompson, 
2007), the significant relationship between the number of 
assessments, poverty level, school size, and student 
achievement is novel and contributes to the knowledge 
base. 
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Figure 3. Scatter plot of the relationship between the number of assessments 
and student performance in the end of the academic year. 


Further, the findings indicate that teachers used the AMC 
Anywhere formative assessment tool more frequently 
for lower-performing struggling learners. Based on this 
finding and data from prior work (Martin et al., 2016), we 
interpreted this finding to mean that teachers gave struggling 
students more frequent assessments to determine if they still 
need more assistance. However, in prior work (Polly et al., 
2014), there were no statistically significant differences 
between poverty level and achievement. 

Implications 

Internet-based technologies continue to be purchased 
by state departments of education, school districts, and 
schools in order to support teachers’ formative assessment 
practices. In this present study, there were statistically 
significant relationships between the use of formative 
assessment practices and  primary-grade students’ 
achievement on number sense tasks using the Internet- 
based tool AMC Anywhere. Further, students from 
impoverished settings, larger schools, and students who 
were assessed more frequently were associated with greater 
growth than their peers. 

While the findings are promising, there is a need for 
further studies. Future research is needed to further detail 
teachers’ specific assessment practices, the frequencies they 
are collecting formative data, and how those data are used 
to plan differentiated instruction in mathematics. State- 
wide, each participating teacher was using standards-based 
mathematics curriculum, as well as DNC resources to teach 
the Common Core State Standards in Mathematics. 
Subsequent studies should examine how the combination of 
specific curricular resources and formative assessment 
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practices influence student achievement on multiple 
learning outcomes such as AMC Anywhere and other 
mathematical tasks. Further, little is known about teachers’ 
day-to-day process of analyzing data and making 
instructional decisions. While AMC Anywhere was used a 
few times a year, more research is needed to examine 
teachers’ daily use of more informal formative assessment 
practices related to number sense as well as other 
mathematics concepts. 

One limitation that we acknowledge is the repetitive use 
of the same assessment. However, we think that growth 
from simply completing the assessment multiple times is 
unlikely. First, the questions in the AMC Anywhere system 
are generated by the tool and are never in the exact same 
order. Second, students were solving questions in a one-on- 
one setting with their teacher, where both the answer and the 
strategy used to find the answer was found. These strategies 
focus on developing fluency and efficiently finding the 
missing part when a number is decomposed. Based on this, 
we are convinced that growth on the assessment includes a 
deeper understanding of decomposing numbers. 

Last, our body of research studies have identified that 
despite professional development and access to AMC 
Anywhere, some teachers do not use the tool much to assess 
their students or they inconsistently enact formative 
assessment practices (Martin & Polly, 2015; Polly et al., 
2014). As a result, there is a need to examine ways to best 
support teachers’ adoption of formative assessment 
practices. For some teachers, the use of an Internet-based 
tool such as AMC Anywhere may be ambitious, and more 
informal, paper-based assessments may be more feasible as a 
starting point for supporting teachers’ use of formative 
assessment processes. There is a need to support and 
research teachers’ adoption of formative assessment 
practices as well as ways to increase their knowledge and 
efficacy related to formative assessment. After all, if teachers 
do not feel capable of doing formative assessment it is 
unlikely to occur, meaning that the growth seen in this study 
will be a missed opportunity in mathematics classrooms. 
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This study focused on science and math professional learning communities (PLCs) that were implemented through a 
university-urban high school partnership. These PLCs were part of mandated school-wide, content-based PLCs 
implemented as part of the reform efforts initiated in an urban school to address the school’s failure to meet Adequate 
Yearly Progress (AYP) for four years consecutively and low graduation rate (less than 25%) for male students. The key 
issues were (a) students had continually earned low test scores; (b) there was continuous principal turnover; (c) faculty 
morale was at an all-time low, and the quality of teaching was very poor; and (d) the students were not effectively 
disciplined. The study examined the impact that university faculty-led mandated PLCs have on teachers’ practices and 
students’ learning and achievement. Analysis of data revealed practices that were effective in developing and 
implementing these successful math and science PLCs. Three themes emerged: ethics of care, teacher agency, and 
aesthetics of professional interactions. Each theme contained key features that appeared to contribute to the 


implementation of a successful PLC. 


Background 

Historically, in the U.S. education may be viewed as 
waves of school reforms described as having the purpose of 
attaining excellence in student achievement and to bridge 
the achievement gap between minorities and their majority 
counterparts (Parker & Parker, 1995). One of these reforms 
was the revision of high school graduation criteria, which 
required students to take more mathematics and science 
courses resulting in a higher percentage of high-school 
graduates completing some advanced coursework in 
mathematics and science (National Center for Educational 
Statistics [NCES], 2004). Despite the increased enrollment 
in advanced-level mathematics and science courses, 
students in the United States, in comparison with students 
in other countries, continue to underperform on 
international standardized math and science assessments 
(Glenn, 2000; Program for International Student 
Assessment, 2012). However, in the lower grades, 
performance in math and science on international tests has 
shown an upward trend (Martin, Mullis, Foy, & Stanco, 
2012; National Assessment of Educational Progress 
[NAEP], 2013). Despite these improvements, students in 
urban schools, who are predominantly Black and Hispanic, 
still perform below their counterparts in suburban schools, 
who are mostly White, Asian, and Pacific Islander 
(Goldberger & Bayerl, 2008; NAEP, 2013). 

The literature on school reform supports and encourages 
the use of Professional Learning Communities (PLCs) as a 
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comprehensive reform initiative (Annenberg Institute for 
School Reform, 2004; Fulton & Britton, 2011). In a PLC, 
stakeholders (teachers, administrators, and parents) 
continuously seek and share learning to increase their 
effectiveness as they reflect on the impact of their actions 
with their students (Leo & Cowan, 2000). 

Professional Learning Community 

A growing body of literature shows that collaboration 
and teamwork practices within supportive PLCs have 
positive outcomes for teachers’ professional development 
(Fulton & Britton, 2011). Collaboration is critical to the 
development of PLCs in schools. However, opportunities 
for teachers to interact either within or outside school have 
been mostly sporadic and random (Maloney & Konza, 
2011). PLCs facilitate the move from the traditional view of 
teachers from being isolated practitioners toward a 
collaborative, learning-centered model (Morrissey, 2000). 
Studies show that PLC embedded professional 
development approaches are effective in improving 
teaching and learning processes (Hord, 2008; Lutrick & 
Szabo, 2012). 

Five components that have been shown to be part of a 
successful PLC are: (a) shared values and visions, (b) 
shared and supportive leadership, (c) collective learning 
and application of learning, (d) supportive conditions, and 
(e) shared personal practice (Hord, 2008; Leo & Cowan, 
2000). The shared values and visions involve an agreement 
by all stakeholders in the PLC about student needs. A 
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fundamental characteristic of the vision is the unwavering 
focus on the students’ learning. The collaboration between 
the stakeholders’ results in creative problem solving, 
strengthens the bond between principals and teachers, 
stimulates learning, and provides opportunities for teachers 
to grow, change, and develop professionally (Hadar & 
Brody, 2012). Shared and supportive leadership involves 
the acceptance of a collegial relationship with others in 
one’s community. This implies an existence of shared 
power and decision making, which is often cited as a 
challenge due to the hierarchical nature of most institutions/ 
school districts. The literature on professional development 
opportunities provided through partnerships is not 
organized in the PLC format. 

Collective learning and its application is a concept that 
has its origin in Senge’s (1990) concept on learning 
organizations as places “where people continually expand 
their capacity to create the results they truly desire, where 
new and expansive patterns of thinking are nurtured, where 
collective aspiration is set free, and where people are 
continually learning how to leam together’ (p. 3). 
Supportive conditions enable the community of learners to 
regularly come together as a unit for the purpose of 
learning, decision making, problem-solving, and doing the 
creative work that characterizes a PLC (Boyd, 1992; Louis 
& Kruse, 1995). 

Peers supporting and learning from each other 
characterize shared personal practice. In such practice, 
teachers regularly visit each other’s classrooms to observe 
and discuss their observations with the observed peer. 
When teachers work together, they share ideas and 
motivate each other and increase their content and 
pedagogical knowledge, which, in turn, improves students’ 
academic achievement (Dierking & Fox, 2012). The 
process is enabled by the mutual respect and 
trustworthiness of the PLC members. 

For a PLC to be effective, learning experiences must 
occur over a period of time, and the members must have the 
opportunity to reflect on the process. Loucks-Horsley, 
Love, Stiles, Mundry, and Hewson (2010) noted that, in the 
past decade, time pressures and balancing responsibilities 
for teachers has been a challenge to science and 
mathematics school reform efforts. 

While there is plenty of research on professional 
collaborative University-School Partnerships, there is little 
research on mandated partnerships and the role of 
university content faculty members in these partnerships. In 
the urban school where the research was conducted, 
participation in the content-based PLCs was mandated. 
Teachers had participated in many types of reform efforts 
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and the mandated PLC participation had the potential to be 
viewed by the school community as the latest “quick fix” 
(Dufour, 2004, 2007) to be endured. Because the mandated 
PLC required a partnership with the local university with 
content faculty members facilitating the implementation of 
the content-specific PLCs, the math and science PLC 
facilitators took into account the tension associated with the 
mandated PLCs. 

Thus, this study examined the impact that university 
faculty members have on science and math teachers in 
mandated PLCs at an urban school. The purpose of the 
study was to answer the foliowing question: What happens 
when science and math university faculty members 
facilitate a mandated school-based PLC that is aimed at 
changing teacher practices and improving student learning 
outcomes? Research on educational change/reforms, the 
role of change agents through reflective practice. and the 
ethics of care inform this research. 


Methodology 
Case Study Research 

Qualitative case study methodology provides tools for 
researchers to study complex phenomena within their 
contexts (Baxter & Jack, 2008). According to Miles and 
Huberman (1994) a case is a phenomenon of some sort 
occurring in a bounded context. The PLCs were confined to 
a particular setting and situation (Clandinin & Connelly, 
2000). A case study method is able to highlight teachers’, 
students’, and the university faculty members’ experiences 
in each of the PLCs (Barone, 2009). Purposeful sampling 
was used in selecting the two PLCs (Bogdan & Biklen, 
1998). Math or science teachers who were grouped into 
content-specific PLCs that had a university content- 
methods specialist comprised the sample. 

A case study method requires the researcher to explore 
and describe natural settings and events (Biklen, 2011). 
Further, when using a case study approach, the researcher is 
open to investigating everything in the setting (Cronin, 
2014). This study involved the exploration of a public 
school and events associated with two PLCs, which were 
considered two cases. A qualitative case study also allows 
the researcher to look at the specific cases in depth. 
According to Merriam (1998), case study methodology is 
utilized “because the researcher wishes to understand the 
particular in depth, not to find out what is generally true of 
many” (p. 208). The researchers took careful notes about 
our work with the PLC teams and recorded information for 
each case. For the entire intervention period, the faculty 
members collected data in three forms: field notes, 
participant observer comments, and artifacts (Bogdan & 
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Biklen, 1998), traditional sources of data for qualitative 
research (Bogdan & Biklen, 2003). 
Field Notes 

Each faculty member wrote detailed field notes during, 
and immediately after each visit with the teachers. The field 
notes were taken to collect details that would be missed 
through the use of audiotapes, such as notes on blackboards, 
seating arrangements, people in the room, and student 
actions. Also, the university faculty members wrote notes on 
what occurred during a meeting, interview, or any other type 
of session. Notes included detailed descriptions of classroom 
activities, reactions, actions, reflections, ideas, and hunches. 
Participant Observations 

Participant observations are the primary source of data 
for this case study. In participant observations, “social 
interaction occurs between the researcher and informants in 
the milieu of the latter, during which data are systematically 
and unobtrusively collected” (Taylor & Bogdan, 1984, p. 
15). The university faculty members were participant 
observers in the weekly PLC individual and collective 
meetings that lasted from one hour to a maximum of four 
hours per week over a two-year time frame. The informants 
in this study were the teachers who were members of the 
PLC teams. The observer’s notes consisted of information 
about specific interactions and follow-up questions that 
arose from these notes. These notes were collected during 
the weekly meetings and, following the PLC team 
meetings, were written in logs. 

As participant observers, the faculty members served 
multiple roles as subject specialist assisting teachers with 
content and pedagogy, coteachers and teachers’ assistants 
performing multiple roles including calling parents and 
guardians about their children’s behavior. 

Artifacts 

Researchers collected artifacts created by teachers and 
students. These included: science laboratory forms, lesson 
plans, drawings created to illustrate concepts, student 
journals and test outcomes, and analyses of the math and 
science tests that the students took. Journal writing 
provided students with a multitude of opportunities to 
enhance their learning. The students were able to: (a) 
integrate their knowledge with their experiences to produce 
new learning, (b) apply theory to practice, (c) engage in 
critical reflection, (d) gain insight into personal 
development, and (e) express their feelings as they learn 
(Heah & Kathpalia, 2010). Each faculty member collected 
at least two artifacts per month. 

Data Analysis 

Data collection and analysis occurred concurrently with 

the triangulation of the three data sources occurring 
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recursively throughout the study. Faculty members checked 
for consistency between data sources within and between 
the PLCs. For example, comparisons were done between 
the lessons plans teachers created with teachers’ classroom 
practices to see if there was consistency. Faculty members 
used data analysis techniques such as pattern matching, 
linking data to propositions, explanation building, and 
cross-case synthesis (Yin, 2013) highlighting the 
convergence of in an attempt to understand the overall. 

Each faculty member completed an analysis of her case 
study and compared the findings. Cross-comparisons of the 
faculty members’ findings that emerged and were agreed 
upon are reported (Bogdan & Biklen, 1998). As Cronin 
(2014) observed, case study analysis involves the “gradual 
building of an explanation and is similar to the refining of a 
set of ideas” (p. 21). 

Credibility and Dependability 

Case study research methodology as a naturalistic 
approach seeks to understand phenomena in context- 
specific settings. Lincoln and Guba (1985) and Yin (2013) 
note that while reliability and validity are essential criterion 
for quality in quantitative paradigms, in qualitative 
paradigms the terms credibility, neutrality or confirmability, 
consistency or dependability, and applicability or 
transferability are to be the essential criteria for quality. 

The research methodologies used provided rich data, 
which enhanced the credibility of the research. In addition, 
triangulation and analysis of data by two faculty members 
increased credibility. Each step of the study is provided in 
great detail ensuring the dependability of the study. Both 
faculty members agreed upon all findings reported. 

Setting 

The local high school at which this project was 
completed is a Title 1, urban school in the southeast United 
States that serves, on average, 614 students. The school 
population consisted of 98% African-American students, 
and 89% of students were eligible for free or reduced-cost 
lunch. The school faced many challenges both before and 
during this collaboration. It was in the process of being 
taken over by the state because it did not meet AYP for four 
consecutive years, 75% of the boys who entered the school 
did not graduate, and the overall graduation rate was 
approximately 33%. Eleven percent of students were 
passing physical science and algebra end-of-course (EOC) 
exams prior to the intervention. Other challenges that the 
school faced included: (a) the perceptions that the school 
was not a safe place to teach, (b) administrator turnover, 
and (c) excessive classroom management issues. 

Information from guidance counselors, parent liaisons 
and the state report card indicated that the students faced 
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several challenges. A large percentage of these students 
lived in high-poverty contexts with an absentee parent(s) or 
guardian(s). Most of the parents or guardians had an 
inflexible work schedule, and many held multiple low- 
paying jobs with no benefits. Many students also worked to 
help to meet the family’s financial needs. Few parents or 
guardians were able to attend parent-teacher-student 
conferences. 

PLC Implementation and Structure 

Both PLCs were single subject with all teachers per 
content area meeting together. The PLC meetings occurred 
every week for 1-2 hours one afternoon a week. After 
school or Saturday meetings also occurred when students 
needed extra help. Both PLCs had a university partner to 
work as a part of the team in the state-mandated reform 
effort to increase students’ learning outcomes. Under these 
conditions, the teachers were reluctant to be part of the 
process. For the first three months, both university faculty 
members spent time listening to the teachers, consciously 
sharing with them teaching strategies and visiting the 
teachers’ classrooms when invited. With time, more 
teachers invited the university faculty members to observe 
them. The faculty members spent time in the classrooms 
teaching and observing how teachers were implementing 
strategies developed and discussed during the PLC 
meetings. 

Both PLCs first established a shared vision and goals. 
Despite the teachers’ prior challenges, they were willing to 
commit to employing strategies that would help to improve 
the students’ performances. They believed that the students 
were capable of learning math and science successfully and 
that they had the content and pedagogical skills to create 
learning environments and opportunities that would cause 
the learning to take place. 

The process of developing the teams consisted of: (1) 
reviewing the students’ data to determine patterns of 
strengths and weaknesses about the content they were to 
learn, (2) working with teachers on lesson planning to include 
specific known pedagogies and subsequently implement 
these practices in their classrooms, (3) identifying chosen 
components of the pedagogies that did not function as well as 
was desired and continued use of the ones that worked well, 
(4) working within the PLC team to select the components 
across the various pedagogies that were most effective and 
developing a highly context/content practice and then, (5) the 
process began again. With each iteration of the process, the 
teams created new materials, forms, and pedagogies that 
better addressed the students’ needs. This strategy 
continuously improved and focused the teacher practices so 
that the students had more opportunity to learn. 
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Participants 

The six math teachers in the department participated in the 
PLC. Their years of experience ranged from 5 to 46. Two of 
the teachers, one of whom was the head of the department, 
had been teaching at the school for over 30 years. Three 
were Visiting International Faculty (VIF). A VIF teacher is 
an international teacher who holds a full-time position in a 
school and gains firsthand knowledge of the United States, 
while helping to prepare students to be global citizens (VIF, 
2015). All but one of the teachers had a master’s degree and 
were categorized as highly qualified as per No Child Left 
Behind Act. All teachers teaching algebra courses for grades 
9-12 participated in the study. Seven science teachers each 
who taught at least one section of physical science 
participated in this study. These teachers had experience 
ranging from 1 to 21 years. They had different certification 
routes, including the traditional route and the Program of 
Alternative Certification for Education (PACE), while others 
were VIF. One of the teachers had a master’s degree and was 
the head of the department. 

Baseline Data and Interventions 

Both PLC teams analyzed data from the High School 
Academic Performance (HSAP) exit exam scores. Students 
must pass HSAP to earn a high school diploma. The 
science teachers used End of Course (EOC) exams. EOCs 
are used to assess content that is not covered by the HSAP. 
Both EOCs and HSAP exams are summative assessments 
created by the Department of Education in the State. The 
interim tests that were used were the Measures of Academic 
Progress (MAP) scores. MAP, a formative assessment, is a 
norm-referenced measure of student growth over time. 
Through the analyses of the data, the teams identified the 
content areas that students had difficulties and developed 
strategies to address students’ learning difficulties and 
challenges. 

In math, the analysis of the MAP and RIT scores showed 
that over 85% of the students lacked basic math skills in 
numbers and operations, algebra, geometry, measurement, 
and data analysis and probability. In addition, analysis of 
student performance on the HSAP exam showed that fewer 
than 5% of the students attempted the word problems. 
Additional data revealed that 89.7% of the students came to 
the high school with performance at or below the basic 
level on their eighth grade math proficiency test. Using the 
analysis of the RIT scores and computer-based math 
assessments, the math teachers and the faculty members 
found that the students had weaknesses in algebra and 
numbers and operations. To address these weaknesses, 
teachers selected learning materials and computerized 
instructional assessments as interventions. 
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Table 1 


High School Assessment Program (HSAP) Exam Pass Rate: Second-Year Students 


Local High School Where Project Was 





High Schools with Similar Students 





Completed 
Percent 2005 2006 2007 2008 2005 2006 2007 2008 
Passed 2 subtests 32.6 47.8 ow), 55.0 53.8 Sail 61.8 64.5 
Passed 1 subtest Sah 29.6 27.9 PEE 22.8 EO 20.8 16.0 
Passed no subtests 38.7 22.6 16.4 17.8 255 S Oa 20.9 20.1 





The results show a steady increase in the number of students who passed the HSAP exam. 


With the combined interventions, there was a marked 
increase in student content knowledge in geometry, algebra, 
and on the general MAP test. Student performance on the 
postintervention MAP test showed significant improvement 
in all of the content standards. Over 50% of the students 
had mastered the standards and needed little or no 
additional interventions. In addition, student scores on the 
HSAP exam continued to increase throughout the period 
when the teachers worked as a team in the PLC. The results 
showed a 50% improvement in the number of students who 
passed the HSAP exam during the duration of the PLC. See 
Table 1. 

In science, MAP testing showed that students had 
difficulty in three areas: (a) focus on multiple variables 
included in a concept, rather than one single 
independent variable that can be controlled in the 
system; (b) had not developed strong scientific language 
abilities; and (c) were not able to interpret the meaning 
of graphs or ways to apply them. Two students scored a 
40% on the pretest. Their scores ranged from 15 to 40%, 
with an average pre-test score of 26%. At the end of the 
intervention, a total of 24 students earned a score of 
70% or higher on their end-of-course exam, 23 earned a 
score of 60-69%, and the remaining students scored in 
the 50% range. 

In science classrooms, after a 15-week intervention 
period, 70-92% of students/class were passing the EOC 
exam. Prior to the PLC, the average pass rate was 11%. In 
cases where students had special needs, the PLC had a less 
dramatic impact but in classes with high numbers of 
students in special education, the pass rate of the EOC 
exam increased from 6 to 40-50%. 

Goal Setting 

Based on the analysis of the summative and formative 
assessment scores, the teams created goals that were much 
deeper than the singular state-mandated goal. The 
overarching goals for both teams were the development of a 
deeper understanding of (a) content-specific epistemology, 
(b) pedagogy and, (c) the culturally relevant practices that 
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valued ways of thinking that the students bring to the 
learning of science and math. 
Content Specific Epistemology 

At the end of the first 15-week period of the intervention, 
teachers began talking about the epistemology of their 
content area. For example, one of the science teachers 
teaching a unit on electricity understood that the students 
were not doing well in their labs because they were not able 
to identify a single variable and test it. Because of this, the 
students were not developing a way to test any items on the 
circuit. During the discussions, it became clear that two of 
the teachers had the same problem as the students. The 
teachers were willing to reflect on their epistemology to see 
if it was consistent with doing science and change their 
misconceptions. In the case of mathematics, one teacher 
whose students demonstrated a deeper understanding of 
how to form and discern math patterns and relationships got 
the opportunity to explain that epistemological component 
of doing math. 
Content Specific Pedagogy 

Over a period of time, the teachers’ epistemology 
gradually became consistent with that of science and math 
and began creating pedagogies consistent with the 
disciplines. In science, the teachers examined the artifacts 
that the students created (test responses, lab results, 
drawings, and dialogue) and developed a lab format 
tailored to their students. The lab format took a holistic 
approach, with embedded items upon which to focus, such 
as independent variables, specific procedures used, and 
generic questions. The teachers developed the generic 
questions into specific questions for the lab used on that 
day. These specific questions were used to develop 
procedures and to help students understand concepts such 
as tables, graphs, and data analysis. The format enabled the 
breaking of concepts into component parts, helped students 
to learn scientific language, and allowed for the asking of 
questions and development of higher-level thinking. This 
approach increased students’ ability to interact with each 
other and to be cognitively engaged in the scientific 
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process, enabling the development of a deeper conceptual 
understanding rather than a simple memorization of facts. 
These changes were observed during classroom 
observations and in students’ artifacts. 

In math, the teachers examined the artifacts that the 
students created (test responses, classroom notes, reflective 
journals, and dialogue) and developed lessons _ that 
promoted conceptual understanding of mathematics. The 
focus shifted from arithmetic to doing math. For example, 
the teachers embedded more word problems in their lessons 
and provided the students with strategies that required them 
to apply appropriate problem solving techniques such 
identifying the problem, developing strategies for solving 
the problem (drawing pictures, charts, tables), 
implementing the strategies and checking their answers. 
The teachers began to emphasize the process rather than the 
final answer. 

Because the teachers cared and were deeply committed to 
their students’ learning, they planned additional time to 
work with their students. The math teachers planned 
Saturday tutoring and two high school exit exam retreats for 
students who were about to sit for their HSAP exams. All 
the students, twenty of them, six parents/guardians, 
guidance counselors and one administrator attended the 
retreat to show their support of the students and importance 
of the exams. Both the science and math teachers provided 
after school tutoring four days a week because they 
believed the students were capable of doing math and 
science but needed more time and practice than was 
available during regular class time. 

Both PLC teams planned for peer observations and video 
recording of their teaching to aid in self-reflections. The 
following questions aided their self-reflection: (a) was my 
content aligned to the disciplines’ epistemology, (b) did I 
express the content well, (c) did I teach in a manner that 
was consistent with my discipline, (d) how did my students 
respond, (e) what did my students learn, and, (f) what is my 
evidence of their learning? 

Culturally Relevant Teaching 

Both PLCs learned how to help the students code switch 
between their home dialect and the math and science 
content. For example, the students’ dialect includes “is” or 
“4s not” but not “might” or “could” (Blake & Van Sickle, 
2001). This language makes understanding of hypothesis, 
conjectures and reasoning difficult for them (Aina, Van 
Sickle, & Blake, 2002). Also, the students have limited use 
of prepositions tending to overuse the word “by” making it 
nonspecific. The teachers learned to use local examples that 
come from the students’ everyday lives. For example, when 
studying pressure laws in chemistry, students were asked to 
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think of what happened to unopened potato bag in a hot car. 
When studying slope, we started with slope zero because 
our landscape is flat. In order to understand greater or less 
than symbols, we discussed the students’ pay scales. These 
examples showed linkages between the context and the 
content they needed to learn. 

Investigating the contributions of minorities in math and 
science was fascinating to the students. The students were 
surprised to find significant contributions in math and science 
by African Americans and other minorities. Journal writing in 
math and note booking in science was particularly exciting 
for the students because the guidelines were flexible, and 
creativity was allowed to demonstrate their understanding of 
concepts. This graded activity forced the students to revisit 
the content that they had learned and to write about it. 
Although the students disliked writing, all completed this 
activity. Journal writing served an unexpected purpose of 
promoting dialogue and rapport within the learning 
community, and as an avenue for the teachers, students, and 
the faculty members to talk about math proficiency in a 
nonjudgmental manner. The students talked about their 
challenges in math and provided the teachers with insight on 
how to differentiate instruction for their benefit. 


Discussion 

In both PLC case studies, there were changes in teacher 
epistemology, pedagogy, and culturally relevant practices 
that had a positive impact on student learning. University 
faculty members’ participation in the PLCs enhanced teacher 
effectiveness in the classroom and improved student 
learning. Three major themes emerged from the analysis of 
the two case studies. The first theme is the ethics of care. The 
participants expressed elements of care. The second theme is 
teacher agency. Together, the university faculty members 
and the teachers in the PLCs created inquiry-based 
instructional materials. The third theme is aesthetics of 
professional interactions. The teachers within each PLC 
began interacting in respectful and meaningful ways. In the 
next section, these three major themes are described using a 
logic that constructs the model for university faculty 
members’ participation in a high school PLC. 
Ethics of Care 

Ethics of care involves attending to the people in a certain 
context. As Noddings (1988) stated, “From the perspective of 
caring, the growth of those cared for is a matter of central 
importance” (p. 221). In both case studies, the university 
faculty members learned to listen and to discern the teachers’ 
perspectives of their students’ abilities and were able to 
provide the teachers with materials, practices, and ideas that 
had the potential to generate positive teaching and leaming 
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outcomes. After several weeks of meetings and discussions, 
in each case study, a “brave” teacher would use the suggested 
ideas that would produce a greater number of positive results 
than in his/her previous practice. Eventually, different 
teachers had the courage to try different materials and were 
more successful than they previously had been. The faculty 
members listened to the teachers, and, subsequently, the 
teachers became more willing to listen to each other. This 
model allowed teachers to attend to each other’s growth and 
the growth of their students. The more the teachers sought the 
academic growth of their students, the more the teachers 
sought strategies to improve their pedagogy. At this point, the 
teachers collaborated to create materials and opportunities 
that were targeted to meet their students’ needs. 

Appreciating the strengths of the group 
members. Noddings (1988) emphasizes the importance of 
open dialogue that is mutual and marked by reciprocity as a 
means to arrive at solutions to difficult problems. Once 
teachers began to engage in research-based practices in 
their classrooms and shared their students’ results during 
the PLC, deeper conversations between team members took 
place. These conversations were about what worked, what 
did not work, and what needed to be changed. The 
conversations and sharing of teaching experiences helped 
the teachers to develop trust in each other, and they began 
to request their colleagues to observe them using practices 
that they perceived as effective. They also videotaped 
themselves teaching and shared the videos with each other. 
These peer observations enriched the conversations and 
became another data source about students’ progress and 
their teaching practices. The team members’ growth as 
effective teachers was noticeable, and their conversations 
were reciprocal. 
Teacher Agency 

Understanding one’s position. University faculty 
members took the position that the members of the PLC 
have the ability to effect change or act as a change agent. 
Gardiner defines an agent as “someone who _ is 
contemplating an action, has already acted, or is presently 
acting” (cited in Bartky, 1995, p. 178). Once the teachers 
were successful, they began to demonstrate these 
characteristics. Through asking questions of the teachers 
and expecting explicit answers, teachers developed the 
capacity to determine and act as agents/knowers, not only 
in their classrooms but also in their PLC. This knowledge 
allowed the teachers to restructure their thinking about their 
ability to cause change. Through listening and observing, 
the teachers were able to discern their students’ 
competencies that were not measurable on standardized 
tests. They began to target the components of the 
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standardized tests that the students needed to be competent. 
This process is what Schon (1991) refers to as reflective 
practice. Teachers recognized that every classroom was 
different and “that strategies must constantly be changed, 
invented, reconceptualized to address each new teaching 
experience” (Hooks, 1994, pp. 11-12). 

Exploring the possibilities within one’s position. Schon 
(2001) notes that, “when a practitioner sets a problem ... he 
names the objects of this attention and frames them in an 
appreciative context which sets a direction for action” (p. 
7). The teachers, in exploring the possibilities of their new 
position within the PLCs, began to voice their concerns 
with the rest of the PLC participants and were able to 
develop a constructive dialogue. In addition, teachers 
became more comfortable and began to develop their 
agency in both PLCs. Out of that teacher agency, some 
took leadership roles within their corresponding PLCs and 
others became more comfortable receiving advice from 
both the faculty members and the lead teacher(s). 

Making change a possibility and engaging in reflective 
practice. The researchers facilitated opportunities for the 
teachers to become agents of change by encouraging and 
creating reflective practitioner opportunities. The teachers 
took responsibility for their own learning (professional 
development or pedagogical knowledge) as well as for 
student learning. These characteristics of a reflective 
practitioner contribute to effective leadership and are a 
major component of change. 

Collaboration between teachers was further promoted by 
the sharing of teaching videos. Faculty members shared 
videos of themselves’ teaching, and encouraged teachers to 
videotape and reflect on their teaching practices. Through 
watching their videos, teachers obtained peer feedback and 
self-analysis. The teachers not only identified and reduced 
habits that minimized their effectiveness, but, more 
importantly, they also recognized and supported those 
habits that maximized their teaching success (Osterman, 
1990). 

Aesthetics of Professional Interactions 

Mutual vulnerability. Hooks (1994) States, 
“Empowerment cannot happen if we refuse to be 
vulnerable while encouraging students to take risks” (p. 
21). All members of the PLC showed their vulnerability. 
Mutual vulnerability was crucial to creating a reciprocal 
relationship between teachers, students, and faculty 
members. The faculty members demonstrated their 
vulnerabilities by modeling the strategies that were 
developed through the PLCs in the teachers’ classrooms. 
The teachers saw the instances when the faculty members 
were successful at teaching and instances when they were 
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not. This encouraged the teachers to take risks by 
implementing new strategies with their students. This 
mutual vulnerability not only provides for an environment 
in which its participants grow but also aids in creating 
teacher empowerment and agency (Hooks, 1994). 

Tactful interactions related to solutions. Because the 
teachers were aware that the school was at risk of state 
takeover and had experienced interventions by other 
outside parties, the thoughtfulness and tact on the part of 
the faculty members became important characteristics. The 
phenomenology of thoughtfulness and tact opened doors 
for relationships in settings otherwise predisposed to 
complicated interactions (van Manen, 2007). 


Conclusion 

The university faculty members played an important role 
in facilitating the development of the five key components 
of a successful PLC: (a) shared values and mission, (b) 
shared and supportive leadership, (c) collective learning 
and its application, (d) supportive conditions, and (e) shared 
personal practice. Of equal importance was the manner in 
which the enactment of these key components was 
negotiated in each PLC and the role that the university 
faculty member played in facilitating these negotiations. 
The university faculty initiated compensatory approaches 
that were constantly focusing on the shared vision and 
values, which were coupled with the strong belief in the 
students’ ability to learn as a central focus of the entire 
process. 

Space was created for the participants of the PLCs to hear 
each other, reflect, and make decisions on classroom 
practices that worked in their context. At the beginning of 
the study, the PLC lacked all five components of a 
successful PLC. The teachers were frustrated by the top- 
down approach of the school’s administration. The 
university faculty members instituted a different leadership 
style, one that incorporated the shared leadership 
component. The PLC did not have a shared vision or goal 
prior to the university faculty members’ intervention, and 
there was no collaborative effort within the group. The 
combination of the three tenets enacted consistently over 
time by the university faculty members produced a 
successful PLC. 

The teachers’ collective learning and application of 
learning contributed to positive results with the students as 
observed in the classroom. These results reinforced 
teachers’ confidence and leadership skills within the PLCs 
and in their classrooms. Teachers’ willingness to discuss 
their successes and failures within their pedagogy exposed 
their mutual vulnerabilities, which seemed to increase the 
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teachers’ trust for each other and to build stronger 
relationships with each other as they engaged in becoming 
agents of change. The teachers’ practices improvement 
correlated with students’ learning and achievement. 


Limitations of the Study 
A major limitation of this study was that this PLC was 
state mandated. The circumstances under which these PLCs 
were implemented were unlike many reported in other 
studies of PLCs. 


Implications of the Study 

The results indicate that, while the development of the 
five components of a PLC is important, each PLC is 
unique. Conditions must be created through which the team 
members can individually and collectively make decisions 
that empower them to become agents of change. The 
presence of a faculty member as a subject specialist 
mandated PLCs, can facilitate the negotiations that take 
place within the PLC. These negotiations stimulate the 
development of the five components of a successful PLC 
discussed earlier. The involvement of faculty members in 
PLCs also can initiate the development of a positive 
dynamic that motivates PLC members to begin the process 
of positioning themselves as agents of change. University 
faculty members can serve as a catalyst to change and 
enhance the likelihood of success in PLCs. 

The three themes of ethics of care, teacher agency, and 
aesthetics of professional interactions were manifested as a 
result of the presence of a university faculty member. An 
external person can ensure that these negotiations occur and 
can facilitate the discussion of difficult issues. These 
interactions enabled the teachers to enact pedagogies in 
their classrooms that generated positive change. 
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Metaphors are regularly used by mathematics teachers to relate difficult or complex concepts in classrooms. A complex 
topic of concern in mathematics education, and most STEM-based education classes, is problem solving. This study 
identified how students and teachers contextualize mathematical problem solving through their choice of metaphors. 
Twenty-two high-school student and six teacher interviews demonstrated a rich foundation for these shared experiences 
by identifying the conceptual metaphors. This mixed-methods approach qualitatively identified conceptual metaphors via 
interpretive phenomenology and then quantitatively analyzed the frequency and popularity of the metaphors to explore 
whether a coherent metaphorical system exists with teachers and students. This study identified the existence of a set of 
metaphors that describe how multiple classrooms of geometry students and teachers make sense of mathematical problem 
solving. Moreover, this study determined that the most popular metaphors for problem solving were shared by both 
students and teachers. The existence of a coherent set of metaphors for problem solving creates a discursive space for 
teachers to converse with students about problem solving concretely. Moreover, the methodology provides a means to 


address other complex concepts in STEM education fields that revolve around experiential understanding. 


In the last 60 years, mathematics educators have 
perceived mathematical problem solving as a heuristic 
process (Pélya, 1945), a logic-based program (Newell & 
Simon, 1972), a means of inductive and deductive 
discovery (Lakatos, 1976), a framework for goal-oriented 
decision making (Schoenfeld, 1985, 2010), methodologies 
with multiple variables (Kilpatrick, 2004), a standard 
(NCTM, 1989), and a model-eliciting activity (Lesh & 
Zawojewski, 2007). In their historical analysis of problem 
solving, Stanic and Kilpatrick (1989) state, “The term 
problem solving has become a slogan encompassing 
different views of what education is, of what schooling is, 
of what mathematics is, and of why we should teach 
mathematics in general and problem solving in particular” 
(p. 1). Consequently, definitions of problem solving can be 
nebulous, or downright conflicting (Schoenfeld, 1992). As 
Shumway (1982) states, “We (researchers of problem 
solving) don’t know what problem solving is, but if you 
read our definitions you'll know what it isn’t” (p. 131). 

Despite these varying perspectives on problem solving, 
students are told problem-solving skills are vital for their 
secondary mathematics education (NCTM, 2000; NGA & 
CCSSO, 2010). Secondary school teachers have significant 
influence over students’ perception of problem solving (Polya, 
1945; Schoenfeld, 1985). To communicate in a language that 
is meaningful to the student, it has been suggested that a 
teacher could start with the student’s contextualization of 
problem solving rather than assuming the student shares the 
teacher’s meaning of mathematical problem solving 
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(Sakshaug & Wohlhuter, 2010). However, current research 
lacks a means to tap into students’ experiences that generate 
students’ contextualization of problem solving. 

The goal of this study is to understand secondary students’ 
and teachers’ contextualization of mathematical problem 
solving. Understanding how students and _ teachers 
contextualize problem solving can give researchers and 
practitioners a concrete starting point from which to generate 
productive discourse about problem solving, despite the 
cacophony of definitions. Accomplishing this feat requires 
significant focus on the language and experiences of students 
and teachers. Students rely on their experiences to make 
sense of mathematical concepts. With problem solving, the 
experiential complexity stems from interpretation and 
communication (Pélya, 1945). To align the researcher’s 
interpretation with the student’s and teacher’s experiential 
perception, the linguistic tool of metaphor is ideal (Sfard, 
1998, 2000) because metaphors allow people to relate 
experiences for communication and cognition (Presmeg, 
1997). To identify how secondary students and teachers 
contextualize mathematical problem solving, this 
interdisciplinary study answers the following questions: 


Q1: What metaphors do secondary students and 
teachers use to describe their experiences with 
mathematical problem solving? 

Q2: Is there a coherent subset of these identified 
metaphors that can inform research on how students 
and teachers perceive problem solving? 
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To answer these questions, this study collected data from 
seven high school geometry classes via 28 student and 
teacher interviews. Each interview had students or teachers 
express their perceptions of a specific mathematics problem 
from a common assessment. The data were analyzed using 
a mixed methods design incorporating Conceptual 
Metaphor Theory (CMT, Lakoff & Johnson, 1980). A 
qualitative phenomenological analysis identified the 
conceptual metaphors used by students and teachers for 
mathematical problem solving. A quantitative analysis used 
the qualitatively identified conceptual metaphors to 
determine the existence of a coherent subset of metaphors 
via frequency and popularity. This study found that such a 
coherent subset of metaphors existed and had significant 
overlap between teachers and students. As a result, this 
study posits that identifying coherent metaphors can 
concretize (Danesi, 2007) problem-solving discourse 
between students and teachers. This study offers a working 
set of conceptual metaphors for mathematical problem 


solving to help teachers and researchers practice 
mathematical discourse (Moschkovich, 2007) with 
students. 

Framework 


Mathematical Problem Solving 

To answer the research questions, we need to be able to 
interpret the students’ perception of problem solving. 
Students’ conceptual understanding of problem solving 
should be at the center of all interpretations and applications 
of problem solving (NCTM, 1989, 2000; Schoenfeld, 
1985). Without knowing the student’s perception of 
problem solving, one cannot be sure the teacher’s 
interpretation aligns with the student’s perception. Silver 
(1985) writes: 


In order to give a richer characterization of mathematical 
problem solving, we must go beyond process-sequence 
strings and coded protocols. And we need also to go 
beyond simple procedure-based computer models of 
performance. We need to develop new techniques for 
describing problem-solving behavior not only in terms of 
the procedures utilized but also in terms of the conceptual 
systems that influence performance. The successful 
implementation of rigorous modeling techniques of this 
type would give us more powerful models of problem- 
solving performance ‘and might provide more suitable 
models for learning, since they could account for the 
phenomena of cognitive restructuring and conceptual 
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reorganization that are observed as people learn in 
complex domains. (p. 258) 


This study strives to identify such a cohesive conceptual 
model that will allow for “cognitive restructuring” and 
“conceptual reorganization” (p. 258). Such a model aligns 
with the framework of a cognitively embodied mathematics 
(Lakoff & Nunez, 2000) because students’ understanding 
of mathematics is embedded within their mathematical 
experiences (Nunez, 2008). Hence embodied cognition 
mandates a substructure, a conceptual system (Silver, 
1985), to describe and identify students’ mathematical 
problem-solving experiences. 

Identifying said experiences is complex for research because 
of the layered mediums through which this communication 
takes place. Steffe (1983) reminds us that the experience of the 
student is expressed by the student, interpreted by the 
researcher, related via the researcher’s experience, and then 
expressed by the researcher. To minimize misinterpreting 
students’ perceptions of problem solving requires hermeneutic 
and epistemological clarification of how the researcher will 
discuss mathematical problem solving. 

Hermeneutically, while we cannot assume a definition of 
problem solving within this framework, we can define a 
mathematical problem. Polya (1945) demonstrated the need 
to separate mathematical exercises from problems by 
demonstrating how problems require multiple heuristics 
(i.e., methods of discovery, p. 112) while exercises did not. 
Mayer (1998) described this distinction between “routine 
problems” (exercises or problems students have seen 
before) and “non-routine problems” (problems students 
have not seen before). Nonroutine problems require 
heuristics such as association, abstraction, comprehension, 
synthesis, and generalization which have been theoretically 
framed as metacognition (Schoenfeld, 1992). This study 
defines mathematical problems within the frame of Pélya 
(1945), Mayer (1998), and Schoenfeld (1992) where a 
mathematical problem requires selecting appropriate 
heuristics and not directly applying a previously recognized 
single procedure. This definition of problem emphasizes the 
need for clear mathematical discourse as a means to directly 
recognize which heuristics students select for specific 
problems. 

Mathematical Discourse 

The epistemological perspectives of mathematical 
discourse vary significantly. Pdlya (1945) described 
thinking in problem solving as “mental discourse” (p. 133) 
with oneself. Moschkovich (2007) argued that discursive 
practices involve more than language, but perspectives and 
conceptual knowledge. Many discourse theorists agree that 
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communication is sufficient for constructing and sharing 
knowledge (Herbel-Eisenmann & Cirillo, 2009; 
Moschkovich, 2007), but Sfard (1998, 2000, 2009) claims 
it is also necessary. Sfard (2000) states “Effective 
communication is not explained by the fact that 
interlocutors have the same meanings (or ‘shared 
conceptions’) but rather defines the situation of two 
persons’ ‘having the same meaning’” (p. 300). Sfard is 
arguing that problem solving depends upon the participants 
and their shared experiences. Indeed, Cozza and Oreshkina 
(2013) found students’ metacognitive meaning of problem- 
solving stemmed from collaborative cyclical discourse 
patterns of exploration and implementation with each other. 

Fundamentally, this study’s hermeneutic and 
epistemological framework aligns with Sfard’s (2009) work 
on cognitive discourse because of the focus on shared 
experiences. Despite all the mediums through which one 
must interpret student and teacher perceptions of 
mathematical problem solving, the student and teacher are 
trying to communicate their personal experiences through a 
set of presumed shared experiences. Shared experiences are 
fundamental because the assumption of what students and 
teachers perceive as shared experiences define their 
interpretation of problem solving and how communication 
is demonstrated in the classroom. 
Discourse Analysis via Conceptual Metaphors 

The shared experiences expressed through discourse are 
interpreted and communicated through the use of metaphors 
(Kovecses & Benczes, 2010). Presmeg (1997) reminds us that 
the Greek word, metaphora, means to transfer or carry over. 
The influence of this transfer from a cognitive perspective 
encouraged Lakoff and Johnson (1980) to expand on Reddy’s 
(1979) idea of the conduit metaphor to create the conceptual 
metaphor. In the conceptual metaphor, the literal relationship 
between figures of speech is replaced with a conceptual 
mapping between linguistic expressions (Lakoff, 1993). For 
example the literal metaphor, “Your theoretical framework 
lacks a solid foundation” would have the conceptual metaphor 
THEORIES ARE BUILDINGS! attached to it. Thus with 
each metaphor, two concepts are connected: 


Literal Metaphor: The actual literal expression. 
Conceptual Metaphor: TARGET DOMAIN IS 
SOURCE DOMAIN 


‘Lakoff and Johnson (1980) used capital letters to 
delineate the literal and conceptual metaphors more easily, 
this paper will follow the same format. 
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In all conceptual metaphors, there is a source domain and a 
target domain. The source domain is the experientially 
known domain and the related concept is the target domain. 
In the metaphorical linguistic expression “The solution 
escapes me,” the target domain is solutions while the source 
domain is prey. Hence the conceptual metaphor is 
SOLUTIONS ARE PREY. It is important to note that despite 
the use of the being verb “ARE,” the phrase is unidirectional 
(TARGET DOMAIN — SOURCE DOMAIN). 

Cognitive linguists typically classify these conceptual 
metaphors into three hierarchal categories: structural, 
ontological, and orientational (Kovecses & Benczes, 
2010; Lakoff & Johnson, 1980). Structural metaphors 
strive to describe a complex concept, such as time, in 
terms of a concrete experiential object, such as a limited 
resource, ic., “Don’t waste my time.” Ontological 
metaphors provide target domains with less structure and 
a new reality in which they may be defined. 
Personification is regularly ontological, as in the phrase 
“the solution escapes me.” Orientational metaphors are 
the most difficult to relate experientially according to 
linguists (Kovecses & Benczes, 2010). They are broad 
concepts with only a specific direction inherent in human 
development. The metaphorical expressions “things are 
looking up for him” and “he fell ill” are examples of the 
conceptual metaphor HEALTHY IS UP. These 
categories of conceptual metaphors (Structural, 
Ontological, and Orientational) are used to map the 
communicative and cognitive nature of a person’s 
experiences (Lakoff & Johnson, 1980). 

Mathematics teachers and students share an experiential 
set: solving mathematics problems. However, students’ and 
teachers’ strategies for solving a mathematical problem can 
vary (Lakatos, 1976; Lesh & Zawojewski, 2007; 
Schoenfeld, 1992). Metaphors are culturally designed to 
articulate these implicit perspectives. As conceptual 
metaphors have been used to foster cognition in 
mathematical problem solving (Lakoff & Nunez, 2000; 
Sfard, 1998), this study will use conceptual metaphor 
theory (CMT) to answer our first research question, Q1. 
CMT in Practice 

CMT was introduced by Lakoff and Johnson in 1980 and 
has burgeoned in many fields such as linguistics, cognitive 
science, psychology, education, and neuroscience (Lakoff, 
2008). In CMT analysis, the researcher interprets the 
student model through the mapping of the conceptual 
metaphor. Prus-Wisniowska (1995) used this method 
qualitatively to interpret six undergraduate students’ use of 
metaphors with the concept of mathematical limits. 
Danesi’s (2003) applied CMT to mathematics education as 
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a means to help in understanding concretization, putting 
intangible terms into experiential terms: 


An abstract concept in CMT is defined simply as a 
*“‘mapping”’ or an “‘extension’’ of one domain onto the 
other. This theory suggests that abstract concepts are 
formed systematically through such extensions and that 
specific metaphors are traces to the target and source 
domains. (p. 228) 


This mapping of concepts emphasizes the epistemic view 
of learning as participatory (Sfard, 1998) because it focuses 
on the metaphors as conduits for communication (Reddy, 
1979). Danesi (2007) used CMT analysis on eighth 
grader’s perspectives of mathematical word problems and 
found that their ability to concretize time according to the 
source domain of a line, hence the time line and the 
conceptual metaphor of TIME IS A LINE was helpful with 
basic algebra if diagrammed. However, none of these 
aforementioned studies have been able to take the CMT’s 
qualitative method and quantitatively determine a coherent 
set of shared experiences for a given population using 
coherence, which is the purpose of this study. 

To summarize, mathematical problem solving is a 
complex topic in mathematics education and difficult to 
concretize. However, cognitively embodied mathematics 
(Lakoff & Nunez, 2000) requires educators to question how 
students and teachers contextualize and conceptualize 
problem solving if mathematical discourse is allowed to 
help students share and construct meaning (Moschkovich, 
2007; Sfard, 2000). Thus there needs to be a model (Lesh & 
Zawojewski, 2007) that identifies the student’s and 
teacher’s shared and personal experiences of mathematical 
problem solving. This interdisciplinary study will construct 
such a model using CMT analysis (answering Q1) and then 
determine if a subset of- problem-solving metaphors exist 
for students, teachers, or both students and teachers 
(answering Q2). 


Method 

Initially, a pilot study (Yee & Bostic, 2012) interviewed 
nine high school students and asked them to describe their 
problem-solving techniques with three distinct problems 
(Yee, 2013). The data were analyzed using CMT analysis. 
The results of the pilot study demonstrated rich structural, 
ontological, and orientational metaphors _ students 
associated with mathematical problem solving (Yee & 
Bostic, 2014). The pilot study was the foundation for this 
study. 
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Participants 

This study took place at a suburban high school with 
geometry students and teachers. These geometry classes 
were chosen because the culture encouraged students to 
justify their conclusions, which was helpful in being 
thorough in the interview process. Two teachers (Alpha and 
Beta) and 22 ninth- and tenth-grade students volunteered to 


participate in the study. 
Design 

Together, teachers Alpha and Beta designed three 
common assessments. The researcher individually 


interviewed Alpha, Beta, and all participating students 
during the first three common assessments of the geometry 
course. Alpha and Beta were each interviewed three times, 
once with each common assessment. Hence there were six 
teacher interviews and 22 student interviews in total. All 
participants met with the researcher individually for a 15— 
20-minute video interview. While classroom and group 
discussions could yield valuable conceptual metaphors as 
well, such conversations would implicate too many 
variables (i.e., who owned which metaphor) for the primary 
purpose of this study. Student interviews focused on one 
problem from the common assessment that each student 
had already completed (Figure 1). 

Researchers selected problems from the teacher-designed 
common assessments that offered significant discussion of 
their techniques. All three problems were nonroutine 
(Mayer, 1998) because both teachers had not used similar 
problems in homework or classroom exercises and thus no 
single, direct strategy had been shared. For example, in 
Common Assessment 3, students had learned the multiple 
methods of identifying if two triangles were similar, but the 
ideal technique was not directed in the instructions. Of the 
three common assessments, seven students were 
interviewed for Common Assessments, | and 2, while eight 
students were interviewed for Common Assessment 3. 
Thus, there was a balanced distribution among geometry 
students. 

Data Collection 

To make sure the researcher could ask clarifying 
questions to properly identify students’ metaphors, a 
semistructured interview method was used (Table 1). 
Students had already taken the selected common 
assessments and had access to their previous work on the 
assessment during the interview. Researchers watched the 
video and transcribed all conversations. 

After collecting data, the researcher: (1) qualitatively 
coded the audio and video data using CMT analysis to 
determine which source domains students and teachers 
associated with the target domain of problem solving and 
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Common 
Assessment Question # 
1 x 


Problem 


A regular hexagon has rotational symmetry about its center. What is the 


minimum number of degrees of rotation necessary to show rotational 
symmetry? Explain your reasoning using math vocabulary. (No picture 


2 7 Find the area and perimeter of this figure. 14 
10 
14 
6 
8 
3 la and Ic Determine whether or not each pair of triangles is similar. 


Justify each by using a flowchart. 


la. 


Y R A 
; J 
K 


lice 


Z K 
6 
Ho 9 J 
8 
Y 7 
11 
Xx 
L 


Figure 1. Questions from the first three honors geometry common assessments. 


(2) quantitatively analyzed the frequency and popularity of 
each source domain to determine which source domains 
significantly describe mathematical problem solving. 
Qualitative Analysis 

When using CMT analysis, each literal and conceptual 
metaphor was identified and categorized via cognitive 
function (structural, ontological, or orientational). The 
categories were not _ preconstructed, but did 
phenomenologically interpret (Eatough & Smith, 2008) 
students’ and teachers’ literal metaphors. The following 
conceptual metaphor analyses have been shared as 
prototypical examples clarifying the interpretative design of 
CMT to offer insight into how student and teacher 
experiences can be interpreted from their discursive and 
experiential use of problem solving. 

Problem solving is a journey. Students and teachers 
used the source domain of journey pervasively as a 
structural metaphor throughout the interviews. Penelope 
stated that to solve the problem she “followed the 
directions,” the act of following adds structure in her 
approach to problem solving. Oliver’s statement, “I don’t 
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know, that wouldn’t have gotten me too far,” refers to 
distance from the solution, indicating that the theme 
involved is traveling to the solution, i.e., the journey. Phil 


Table 1 

Semi-structured Interview Questions for Teachers and Students 

Teacher Interview Questions Student Interview 
Questions 





How will students perceive the 
mathematical problem? 

Why do you think students will 
perceive the problem the way 
you have described it? 

How will students solve the 
problem? 

Why will students solve the 
problem that way? 

Could students solve the problem 
differently? 


How did you perceive the 
mathematical problem? 
What does it mean to 
solve the problem? 


How did you solve the 
problem? 

Why did you solve the 
problem that way? 
Could you have solved 

the problem 

differently? 
What possible problem solving 

techniques do you see students 

using to solve this problem? 


—_—e_aeererererereer 
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referenced reflection, an important aspect of problem 
solving (Polya, 1945) by stating, “when I look back at what 
I did, I kind of think I might have gotten it mixed up with 
something else.” The action of looking back (as opposed to 
looking forward) is a context generated to express where 
one has been on a journey. Beta also uses the structure of a 
journey when students solve problems because they will 
have to “use the short cut conjectures that we’ve talked 
about,” and Alpha discusses that for one problem “there is 
not really more than one way to go about it.” Thus, we see 
that students and teachers are regularly drawing on the 
structure of journey to contextualize problem solving. 

Problem solving is a race. While students and teachers 
used multiple source domains for problem solving, the 
teacher Beta regularly referenced the structural metaphor 
PROBLEM SOLVING IS A RACE. Beta used statements 
such as “the biggest hurdle that I think that they might run 
into is” and “that is a tough hurdle for a lot of the kids.” 
Beta used the word “run” referencing a fast-paced journey 
such as a race. Hurdles are iconic of races, thus 
corroborating the experiential association with races. This is 
further confirmed when Beta states, “It’s one thing that I 
think might trip up a few students or might at least at the 
beginning.” When one is tripped up at the beginning of a 
race, they may or may not be able to continue. However, 
this metaphor references a shared experience of the source 
and target domain, “the beginning.” When Beta states “at 
least at the beginning” he is implying that the beginning is 
not the end of the problem-solving technique, thus 
implying that the hurdle, or the object that trips up the 
student, impedes the student but does not terminate their 
efforts. 

While Beta was unaware that he was using such 
metaphors, these metaphors carry significant entailments 
(Lakoff & Johnson, 1980) for students who presume that 
problem solving is a race. Specifically, students may 
identify that the speed at which one solves a problem is 
more significant than the quality or understanding of the 
problem. Additionally, the metaphor of race may entail the 
need for competition, to finish the test in a speedy fashion 
as a measure of intelligence. Fundamentally, there seem to 
be significant sociomathematical norms (Yackel & Cobb, 
1996) involved with the choice of metaphors of the teacher. 
These discursive norms can influence students’ structural 
conceptions of problem solving (Sfard, 2000). 

Problem solving is discovery. A verb that both 
students and teachers used regularly was “realize.” Ken 
states, “I knew that was going to be important because 
when two lines were parallel you can realize what angles 
are similar and stuff.” Additionally, Zandra stated that, 
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“when I was using my other method I kind of realized that 
that was completely off because it was over 90 degrees.” 
Teacher Alpha states, “They need to realize that this is not 
going to be part of the perimeter” and Beta states, “I think 
the first thing that students have to realize about part (a) is 
some will look at that and will not really see that there are 
two triangles there.” 

The word “realize” represents an action by the students 
that will make them aware of important information. 
Merriam-Webster Dictionary (2014) defines discovery as 
“to make known or visible,” which is how students and 
teachers are using the word realize. Thinking of problem 
solving as a discovery redefines what constitutes solving 
without adding a firm experiential structure, unlike journey. 
When thinking of problem solving as a journey, there is a 
destination a priori. However, discovery lacks this structure 
because one does not know what one is to discover. Thus, 
discovery offers a varying ontological perspective on 
problem solving because discovery offers some structure, 
but changes the reality of the solution. 

Problem solving is building. Both students and 
teachers referred to the structural metaphor relating problem 
solving to building, aligning with constructivist approaches 
to learning. For example, Oscar referred to the cognitive 
aspect of problem solving when saying, “I put all those 
components together to create one big complex thought.” 
The teachers used the building metaphor in discussing how 
the students learned through inductive reasoning. Beta 
stated, “We tried to build up the pattern showing them 
what’s the degree of rotation of an equilateral triangle.” 
Adison also expressed the building metaphor when stating, 
“They know they need to set up a flowchart” because 
setting up references the foundation with the orientation of 
up. Building was also referenced by teachers and students 
using the verb “made.” Erik stated, “You probably could 
have made it like a trapezoid I guess.” Oliver described, “Tt 
made it a lot more organized than stating facts randomly all 
over.” Zach admitted, “I just did make like, little mistakes.” 
The action of “making” was a regular reoccurrence of 
solving problems. 

Problem solving is a process. Teachers and students 
frequently referred to solving problems as a process, 
structurally emphasizing how one solves problems. 
Moreover, students and teachers literally referenced the word 
process while describing problem solving. Phil stated, “I had 
a different thought process and I’m kind of thinking that these 
can’t equal 360.” Phil perceives problem solving as a 
cognitive process, yet recognizes that there are multiple 
processes. Rhonda referred to her “automatic brain” that 
helped her “find the process then find the solution.” Rhonda 
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continued by saying, “once you got the solution you just 
write down your process. Teachers always want the process.” 
Rhonda’s automatic brain is an enlightening illustration of the 
utility of the process metaphor. Her perspective on teachers 
always wanting to see a process suggests an emphasis on the 
“how” in her geometry classroom. 

Teachers also referred to problem solving as a cognitive 
process. Alpha pointed to the student’s cognitive use of 
problem solving as a process when she stated, “It’s also 
them understanding what notation is there but what 
notation they need to use to explain their thought process.” 
Similarly Beta refers to a specific problem-solving process 
when he states, “I think that they’re a little bit more 
comfortable with the flowchart process and they know to 
get right into the geometry part.” The flowchart process is a 
variation on two-column proofs (Dietiker, Kysh, Hoey, & 
Sallee, 2007). Teachers regularly referenced the “process” 
as the “how” of problem solving for tangible assessment of 
student performance. Both teachers demonstrated the 
“process” through the form of a procedural source domain. 
The procedural perspective of problem solving viably 
addresses the “how” within the problem solving, but omits 
the “why” behind the process. 

Quantitative Analysis 

From the qualitative analysis of 22 student interviews 
and six teacher interviews, this study analyzed students’ 
scores (according to their teacher’s rubric), students’ and 
teachers’ structural conceptual metaphors, ontological 
conceptual metaphors, orientational conceptual metaphors, 


total source domains (per participant), and different source 
domains (per participant). Student and teacher source 
domains associated with the target domain of problem 
solving were then tallied in two different ways. The total for 
each source domain was referenced as the frequency of a 
source domain. However, the total number of times a source 
domain was used (frequency) may not demonstrate how 
pervasive the source domain was among all students. For 
example, it may be the case that Evan used PROBLEM 
SOLVING IS A JOURNEY enough times in his interview 
to compensate for another student not using the journey 
source domain at all. Thus for internal validity, the source 
domains were also analyzed for popularity, the number of 
students who used the metaphor. With popularity, if any 
student or teacher used PROBLEM SOLVING IS A 
JOURNEY during his/her interview, it was only recorded as 
“used” for that student. Together, these two quantitative 
analyses determined if a metaphorical system (Kovecses & 
Benczes, 2010) existed by comparing each source domain’s 
frequency and popularity between teachers and students. 
Teachers’ perceptions of student problem solving using 
conceptual metaphors were compared to _ students’ 
perceptions of mathematical problem solving to identify 
similarities and differences. 


Results 
Qualitative Results 
CMT analysis illuminated relationships between teacher 
and student thinking about problem solving. Figure 2 
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Figure 2. Venn diagram of problem solving source domains for teachers and students. 
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Table 2 
Descriptive Statistics of Students’ Conceptual Metaphors for Problem Solving 
N Minimum 
STUDENT INTERVIEWS 
Score 2a 50% 
Structural Metaphors 2a 8 
Ontological Metaphors oe 0 
Orientational Metaphors 22 0 
Total Source Domains Ze =) 
Different Source Domains oe 3 
TEACHER INTERVIEWS 
Structural Metaphors 6 12 
Ontological Metaphors 6 6 
Orientational Metaphors 6 0 
Total Source Domains 6 11 
Different Source Domains 6 1 


Maximum Mean Standard Deviation 
100% 76.2% 18.5% 
45 17-5 9.10 
18 5.91 Silo 
3 0.59 1.01 
=) 19.14 10.72 
9 6.27 152 
33 22.309 8.28 
10 1.7 1.97 
1 0.167 0.41 
39 26.17 10.53 
10 7.83 2.40 





illustrates the source domains used by students, teachers, 
and both parties. 

Figure 2 shows that teachers used one more source 
domain than the students associated with the target domain 
of problem solving. It is important to note that there are 
slightly more source domains that are shared by the 
students and teachers than there are exclusive to students or 
exclusive to teachers. 

Quantitative Results 

This study coded 157 conceptual metaphors of teachers 
and 421 conceptual metaphors of students that were 
associated with problem solving. Table 2 gives the 
descriptive statistics for the source domains relative to student 


and teacher interviews. Notice that the average score of the 
questions chosen for this study (assessed by the respective 
teacher) was 76.2%. This aligns with a grade letter of C 
which is what the teachers strove to have as the average. 

The average number of conceptual metaphors categorized 
by cognitive function was consistent with linguistics 
research (Kovecses & Benczes, 2010) because Table 2 
shows that structural metaphors occurred the most often 
(student M = 17.55 SD = 9.10, teacher M = 22.17 SD = 
8.28), then ontological metaphors (student M = 5.91 SD = 
5.15, teacher M = 7.67 SD = 1.97), then orientational 
metaphors (student M = .59 SD = 1.01, teacher M = .167 
SD = 41). This corroborates the framework that students 
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Figure 3. Frequency of students’ source domains mapped from the source of mathematical problem solving. All source domains less than 1% were not shown. 
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Figure 4. Percentage of students (22 total) that used each source domain mapped from the source of mathematical problem solving. 


and teachers prefer to talk about problem solving with more 
structured and concrete experiences. 

On average students used 19.14 source domains and 6.27 
different source domains per interview associated with 
mathematical problem solving. Teachers used more 
conceptual metaphors than students with 26.17 source 
domains per interview and 7.83 different source domains. 
While the interviews were kept to approximately the same 
length for students and teachers, this illustrates that teachers 
referenced more metaphors and more diverse metaphors 
more often than students in their interviews on constructing 
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Figure 5. Frequency of teachers’ source domains mapped from the source of 
mathematical problem solving. All source domains less than 1% were not 
shown. 
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and assessing student understanding of problem solving 
with the selected common assessments. 

To better understand the descriptive statistics of student 
source domains relative to the target domain of problem 
solving, Figure 3 demonstrates the frequency of each source 
domain used by all students and Figure 4 demonstrates how 
popular each source domain was with students. 

When comparing Figures 3 and 4, there is agreement in 
the most frequented and most popular source domains used 
by students associated with the target domain of problem 
solving: JOURNEY, SEARCHING, VISUALIZATION, 
DISCOVERY, PROCESS, BUILDING, PARTITIONING, 
EXPERIMENTING, and REVIEWING. 

The process was repeated with teacher source domains. 
Figure 5 demonstrates the frequency of each source domain 
used by all students and Figure 6 demonstrates how popular 
each source domain was with students. 

When comparing Figures 5 and 6, there was agreement 
in the most frequented and most popular source domains 
used by teachers associated with the target domain of 
problem solving: JOURNEY, DISCOVERY, BUILDING, 
VISUALIZATION, PARTITIONING, SEARCHING, 
PROCESS, and SETS OF SKILLS. 

Comparing the most frequented and most popular source 
domains of both students and teachers yielded a coherent set 
of source domains to contextualize problem solving. Table 3 
lists these overlapped source domains and _ provides 
prototypical examples. It is important to note that 
JOURNEY was the most frequented and most popular 
source domain referenced, while the order of the remaining 
top source domains varied between participants. 
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Figure 6. Percentage of teachers (per interview) that used each source domain mapped from the source of mathematical problem solving. 


Nonetheless, we see the same top seven source domains 
were frequented and popular with both students and teachers. 


Discussion 

This study began by asking what metaphors students and 
teachers use to describe mathematical problem solving 
(Q1). This question was answered using a qualitative 
phenomenological method known as CMT analysis. Using 
qualitative and quantitative analyses, a coherent subset of 
conceptual metaphors was identified and quantitatively 
verified (Q2). Seven source domains dominated the 


interviews among teachers and students in popularity and 
frequency: PROBLEM SOLVING IS A JOURNEY, 
DISCOVERY, SEARCHING, BUILDING, 
VISUALIZATION, PROCESS, and PARTITIONING. 
Other encouraging quantitative results included 
corroborative evidence that the linguistic hierarchy of 
structural, ontological, and orientational conceptual 
metaphors was preserved when discussing mathematical 
problem solving (Kovecses & Benczes, 2010; Lakoff & 
Johnson, 1980). Our results supported Danesi’s (2007) 
results that students concretized problem solving more 


Prototypical Literal Metaphors 


“T don’t know, that wouldn’t have gotten me too far.” 
“We haven’t gone over this yet so I don’t know the specifics.” 
“When I was using my other method I kind of realized that that was completely 


off because it was over 90 degrees.” 


“Once they realize that that’s six sides then they’ll see that we’re looking for 


“Since there was a triangle there I knew that if I found it, the full length of it, I 


could do the Pythagorean Theorem.” 


‘My students would have to go searching for them [the solutions].” 
“T know that if I add them together it would equal the same amount of different 


figures put together because it’s just them separated.” 


“Dividing into parts that you know putting the full parts together to get the total 


“T can’t see all the rules for the one shape in my head.” 
“So I think that they will still try to visualize what’s actually happening as that 


“Tt is the way that my brain processes it that it just flips or doesn’t make sense.” 
“T think that they’re a little bit more comfortable with the flow chart process and 


they know to get right into the geometry part.” 


“You have to take them apart, that’s what I did, I took them apart in my head.” 


Table 3 
Most Frequented and Popular Teacher and Student Source Domains with Exemplars 
Source Domain Person 
A JOURNEY Student 
Teacher 
DISCOVERY Student 
Teacher 
rotational symmetry.” 
SEARCHING Student 
Teacher 
BUILDING Student 
Teacher 
amount” 
VISUALIZATION Student 
Teacher 
hexagon is rotating.” 
A PROCESS Student 
Teacher 
PARTITIONING Student 
Student 


“T solved them individually it’s a lot easier than having to solve it all at once.” 
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frequently with structural conceptual metaphors than 
ontological or orientational metaphors. 
STEM Extensions of This Study 

This study focused on mathematical problem solving. 
However, there is no reason this study must remain bound 
to mathematics. A viable expansion would be to apply 
CMT analysis in other STEM fields. Physics education may 
find similar or different source domains when physics 
problems are discussed with teachers and _ students. 
Ultimately a large-scale study could look to problem 
solving across multiple STEM disciplines to find sets of 
coherent structural metaphors and shared experiences. 
Implications for Practice 

It is vital that practitioners understand that only the 
existence of conceptual metaphor systems has been 
identified with this study, not uniqueness. In Joseph 
Schwab’s (1970) classic, “The practical: A language for 
curriculum,” Schwab distinguishes between theoretical 
inquiry and practical inquiry. Theoretical inquiry’s purpose 
is comprehension while practical inquiry’s purpose is 
making a decision in a particular situation. Despite different 
purposes, the overlap of practice and theory is what Lakoff 
and Johnson (1980) identified as coherence. Rather than 
looking for an absolute metaphorical system or excepting 
that no two systems are identical, coherence desires a 
system which will be close enough theoretically to be 
beneficial practically. To this end, this research study 
should not be referenced to demonstrate the uniqueness of a 
metaphorical system, but rather existence. What this 
research offers is a starting point, a possible set of shared 
experiences for teachers to try to use in developing a 
discursive space within their classroom when discussing 
mathematical problem solving with their students. 

Similar to Cassel and Vincent’s (2011) preservice 
elementary teachers’ metaphors for teaching, this study’s 
coherent conceptual metaphors offer teachers experiences 
to explore with students who are struggling when solving 
mathematics problems. Table 3 offers teachers a set of 
experiences that students may likely share in solving 
problems. For example, a teacher may start with the 
JOURNEY source domain by saying, “when I read the 
problem, I was just as lost as you were, but to help me find 
my way, I slowed down and tried to get my bearings by 
determining what would solve the problem, where I 
needed to go.” In this manner, the imagery (Reynolds, 
1993) and experiences of JOURNEY are shared and offer 
a language from which the student is familiar. Moreover, 
this use of JOURNEY can aid teachers because it provides 
a metaphor that can help students “Make sense of 
problems and persevere in solving them” (MP1, NGA & 
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CCSSO, 2010) with a focus on the journey and not the 
destination. 

In its most basic pedagogical form, CMT allows teachers 
to be aware of the metaphors they use while teaching. For 
example, if a teacher is listening and identifies students’ 
literal metaphors referencing the conceptual metaphor of 
PROBLEM SOLVING IS A RACE, the teacher can diffuse 
the situation stating, “I don’t want to think of problem 
solving as a race. While you may have to move fast on 
tests, speed does not indicate your ability to be thorough or 
that you understand the material.” Thus, CMT can offer 
practitioners a method to listen more meaningfully and be 
conscious of the metaphors that influence student learning. 
Implications for Research 

Historically, mathematics education has struggled to 
identify an accepted definition of problem solving 
(Schoenfeld, 1992; Shumway, 1982; Silver, 1985; Stanic & 
Kilpatrick,1989). This study found seven source domains 
that give a coherent framework from which to describe 
problem solving that were meaningful and contextual to 
students and teachers of this study. While there may not be 
a specific definition given to problem solving that research 
communities will agree upon, these seven source domains 
offer research a concrete set of fundamental experiences 
associated with problem solving. 

CMT analysis provided mathematics education with a 
model (Lesh & Zawojewski, 2007) to interpret students’ 
perceptions and interpretations of problem solving. This 
model’s purpose offered a flexible methodology under 
which experiences are shared and summarized using 
conceptual metaphors. This methodology has expansive 
possibilities for research, but it should be made clear that 
CMT analysis needs to be limited to certain topics. 
Mathematical problem solving was an ideal target domain 
because of its multitude of definitions, yet grounded 
experiences and contexts. Marcel Danesi (2007) reminds us: 


The dilemma in education, it would seem, has always 
lied specifically in the teaching of abstract concepts (not 
concrete ones). It is precisely in attempting to resolve this 
dilemma that CMT presents itself as a potentially useful 
framework because it suggests that abstract concepts can 
be concretized in exactly the same way as concrete ones 
such as positive and negative numbers. (p. 228) 


This concretization mandates the need for an embodied 
cognition and thus only useful with topics of less tangible 
understanding that require shared experiential 
understanding. Future topics to be studied using CMT 
analysis could include mathematical generalizations, proof, 


Volume 117 (3-4) 


Mathematical Problem-Solving Metaphors 


and reasoning. Thus this study also offers a novel approach 
to CMT analysis, to interpret the perceptions of teachers’ 
and students’ experiences. 
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Nationally, there is a steadily increasing emphasis on the improvement of STEM education. This includes the 
integration of STEM subjects that have been traditionally taught separately, making it critical that prospective STEM 
educators are equipped to teach using integrated STEM approaches. Connected, an important challenge is providing 
preservice STEM teachers with experiences in which they can develop an understanding of how to optimize learning 
through integrated STEM instruction. A potentially effective way to foster this conceptualization is through video analysis 
of integrated STEM practices. To investigate this possibility, here we present a semester-long study focused on engaging 
preservice STEM teachers with observing, analyzing, and reflecting about instructional STEM practices through a video- 
based intervention. Findings suggest that viewing and reflecting on integrated STEM practices may enhance preservice 
STEM teachers’ conceptions of integrated STEM approaches, representing a practical method of preservice STEM 


teacher professional development. 


Recently, the President’s Council of Advisors on Science 
and Technology (PCAST) recommended that 100,000 new 
Science, Technology, Engineering, Mathematics (STEM) 
teachers be prepared with strong teaching skills and content 
knowledge by 2020. Moreover, current reports by the 
National Research Council and National Academies of 
Engineering have called for shifts in approaches toward 
integrated STEM instruction (NAE & NRC, 2014; NRC, 
2013). Integrated STEM addresses the need for both 
explicit and intentional integration of STEM subjects, 
emphasizing cross-cutting concepts, core disciplinary 
concepts, and viewing science as a set of practices (NGSS 
Lead States, 2013). It has been shown to enhance students’ 
problem-solving skills and science content knowledge 
(Wendell & Lee, 2010). Thus, numerous curricula, models 
of enactment (challenge-; project-; problem-; design-based 
learning), and professional development models have been 
developed (Guzey, Tank, Wang, Roehrig, & Moore, 2014; 
Brophy, Klein, Portsmore, & Rogers, 2008; Cunningham & 
Hester, 2007; Sun & Strobel, 2013). 

However, integrated STEM instruction presents various 
classroom challenges, leading to a resistance toward 
implementation (Liu, Carr, & Strobel, 2009). Along with a 
cited lack of time, resources and support (Lee & Strobel, 
2010), and a range of integrated STEM conceptions held by 
teachers (Radloff & Guzey, 2016), adoption of integrated 
STEM requires adapting current science instruction toward 
this more novel, interdisciplinary approach (Cunningham, 
2008). This means recognizing and utilizing integrated 
STEM approaches, requiring teachers possess adequate 
conceptions of integrated STEM (as pedagogy; a set of 
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disciplines and associated content) supported by effective 
learning opportunities. Yet, often integrated STEM 
instruction is not an intentional part of teacher education 
programs, and most preservice and in-service teachers 
lack adequate integrated STEM exposure (O’Brien, 
Karsnitz, Sandt, Bottomley, & Parry, 2014). 

As an effective, practical means of STEM exposure and 
teacher development, we investigated whether simply 
viewing, analyzing, and reflecting on a set of short video 
examples of effective integrated STEM teaching would 
allow preservice teachers to gain a deeper understanding of 
integrated STEM. While viewing, analyzing, and reflecting 
on videos of teachers’ own and others’ teaching has 
been shown to generally enhance conceptions of teaching 
practices and classroom interactions (Tripp & Rich, 2012), 
studies focusing on integrated STEM teaching and learning 
are scarce. However, videos represent a_ practical 
pedagogical tool for teacher educators, as well as a much- 
needed source of potential development for preservice 
STEM teachers. We focused on the following question: 
How does repeated observation, guided analysis, and 
reflection over a series of short integrated STEM videos 
affect preservice elementary teachers’ conceptions of 
integrated STEM? 

Integrated STEM Education 

Exemplifying integrated STEM, Moore, Johnson, and 
Peters-Burton (2015) provide the following definition: “the 
teaching and learning of the content and practices of 
disciplinary knowledge which include science and/or 
mathematics through the integration of the practices of 
engineering and engineering design of relevant technologies” 
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(p. 24). Herein, the authors discuss six central components of 
every integrated STEM unit or activity, including: 
(a) relevant and engaging contexts, (b) engineering design 
challenges, (c) elements of failure and redesign, (d) 
standards-based math and/or science aims within real-world 
problems, (e) student-centered teaching approaches, and (f) 
an emphasis on teamwork and communication abilities. 
While we acknowledge no integrated STEM conception is 
“better” or “best,” this definition provided an informed basis 
for integrated STEM learning. It also suggested tangible, 
easily visible examples of both integrated STEM content and 
pedagogy for viewing in videos. 

Video-Based Interventions 

Previously, video-based interventions have been effective 
for reflecting on teacher practices and classroom interactions 
(Abell, Bryan, & Anderson, 1998; Bryan & Recesso, 2006; 
Copeland & Decker, 1996; Rich & Hannafin, 2009), 
increasing effective teaching behaviors (Hougham, 1992), 
gaining new classroom perspectives (Sherin & van Es, 
2005), and identifying gaps between teaching beliefs and 
actual practices (Manzi, Preiss, Flotts, & Sun, 2008). One 
underlying mechanism has been identified as teacher 
noticing, or what teachers focus on in videos they are shown 
(van Es & Sherin, 2002). According to van Es & Sherin, 
noticing consists of three facets: (a) recognizing what is 
important in given teaching scenarios, (b) using contextual 
knowledge to reason about those scenarios, and (c) 
connecting specific aspects or events back to teaching and 
learning principles. 

Over time, what teachers notice has been found to move 
initially from classroom management toward student 
thinking and interactions (van Es & Sherin, 2008). Results 
in the integrated STEM classroom suggest enhanced 
mathematics and science teachers’ ability to both notice and 
interpret significant classroom events, as well as gain a 
deeper understanding of science content and learning 
trajectories (Abell & Cennamo, 2004; Goldman, Pea, 
Barron, & Derry, 2007; Johnson & Cotterman, 2015). Yet, 
as Rodgers (2002) discusses, teachers often perceive 
classroom interactions differently within the milieu of new 
reform. 

Concerning methodological approaches to video analysis 
research, effective studies have summarily consisted of: (a) 
viewing a series of their own or other teachers’ videos of 
classroom instruction; (b) selecting events of focus in the 
video (e.g., student’s mathematical thinking); (c) evaluating 
or interpreting these events (Sherin & Russ, 2014). These 
correlated with viewing, analyzing, and reflecting, and have 
been performed through video clubs, discussion groups, or 
written responses (Bryan & Recesso, 2006; Sherin & Russ, 
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2014; van Es & Sherin, 2008). Time of video, number of 
videos, and scaffolding surrounding video analyses (e.g., 
guided vs. unguided reflections) varied widely (Tripp & 
Rich, 2012). 

Here, we used van Es and Sherin’s (2002) aforementioned 
framework of teacher noticing in combination with effective 
video methodology (observing, analyzing, reflecting—Tripp 
& Rich, 2012) to guide our study. We aimed to understand 
the effects of observing, analyzing, and reflecting on only 
five, short (15-minute) videos (1.25 hours total) depicting 
effective integrated STEM teaching using guided reflection. 


Methods 

Descriptive case study methodology was used in guiding 
this study (Yin, 2013). The participants were two senior, 
Anglo-female preservice teachers in their final semester in 
the Elementary Education program at a large, Midwestern 
research university. They were conveniently located and in 
their last semester at the university where this study took 
place. Other potential volunteers cited “lack of time” as an 
issue. Spanning a semester (16 weeks), participants 
engaged in: (a) a preinterview to gauge perceptions and 
understanding of integrated STEM education, (b) viewing, 
analyzing, and reflecting over five, 15-minute integrated 
STEM teaching videos, and (c) a postinterview. Videos 
were given approximately every 2.5 weeks. 

The sequence of videos consisted of a “control” video 
(shown first and last; a premeasure and postmeasure to 
gauge conceptual changes), and three other video cases. 
The video clips that were used in this study were 15- 
minute, edited clips of videos found online at the Teaching 
Channel public website (https://www.teachingchannel.org/) 
and at the Engineering is Elementary website (www.EiE. 
org). They were: a roller coaster design, heat loss unit, 
bridge building, and hand pollinator design. Camtasia video 
editing software was used to remove any teacher reflections 
in the videos beforehand so as not to influence participants’ 
thoughts. All the videos were aligned with the conception 
of integrated STEM education (Moore et al., 2015), and 
contained students: actively engaged in standard-based 
integrated STEM content and practices; working on real- 
world problems; in small groups; designing, testing, 
evaluating, redesigning and communicating engineering 
solutions; using relevant technology. 

Preinterviews and Postinterviews 

Preinterviews and postinterviews were semistructured, and 
consisted of 10-11 open-ended items (pre, 10; post, 11) 
to understand participants’ conceptions of integrated 
STEM before and after the video intervention (see 
Appendix). The added item in the postinterview was to a 
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question asking participants to self-report on the 
effectiveness of the intervention. 
Video Analysis and Reflection 

Video analysis and reflection were separate tasks. Analysis 
consisted of participants being prompted to analyze four 
aspects of an integrated STEM classroom while watching the 
videos: (a) Teacher Pedagogy (what teacher is doing); (b) 
Students (what students are doing); (c) Assessment; (d) 
Example of STEM Integration (whether or not the current 
activity was an example of integrated STEM approaches). 
If participants saw anything interesting, it was suggested 
that they pause the video and respond to the following 
questions: What do you notice? What is your evidence and 
interpretation of what took place? These prompts scaffolded 
teachers to learn to attend to, reflect on, and reason about 
teaching and learning (van Es & Sherin, 2008). 

After watching a video, participants were provided with a 
series of three guided reflection questions. These questions 
attempted to gain a deeper understanding of participants’ 
changing conceptions in conjunction with the video coding, 
asking: 

1. Did this video provide an example of a lesson/unit 
utilizing STEM integration? How do you know? 

2. What components of an effective integrated STEM 
lesson were represented in this science classroom? Which 
were missing? 

3. How did this lesson/unit differ from what you would 
consider an effective science lesson/unit? If you felt like it 
was effective, explain what the teachers and students did 
that made you feel like it was effective. If you did not feel it 
was an effective lesson/unit, please explain what you would 
do differently. 

Secondary Data Sources 

Participants created five lesson plans to implement 
during student teaching as part of normal classroom work 
(separate from the video intervention). Immediately after 
postinterviews we asked if they would volunteer any of 
these plans, and compared them with Moore and colleagues 
(2015) six main components of an integrated STEM unit. 
Data Analysis and Interpretation 

Our analysis served two purposes. The first was to 
explore the effects of video analysis on teachers’ 
understanding of integrated STEM practices by comparing 
their comments for prevideo and postvideo analysis tasks. 
The second was to see how video analysis support changed 
participants’ understanding of STEM integration. To do so, 
preinterviews and postinterviews were transcribed, and 
teachers’ comments from the video reflections were 
extracted. We then read the data set multiple times, and 
generated assertions individually and collaboratively, 
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repeatedly examining the interview transcripts and 
reflections for instances that supported our assertions 
(Erickson, 1986). For example, we systematically looked 
for confirming as well as disconfirming evidence for the 
assertion about change in participants’ conceptions of 
integrated STEM. Afterward, we performed both within 
and cross-case analyses (Yin, 2013). During our weekly 
research meetings, we discussed, developed and compared 
case descriptions for each participant. 


Results: Preservice Teacher Case Studies 
Case 1: Jill 

Background. Jill was a senior in the Elementary 
Education program at the university where this study took 
place. She had taught English for roughly three semesters 
(through the teacher placement program), and ultimately 
desired to teach at the K—2 level in her home town. Jill was 
in her early twenties, from a middle-class, Anglo-family. 
Not previously interested in integrated STEM education, 
she had not thought about it prior to the first interview 
‘other than one lesson in her elementary science methods 
course’ in the previous semester. When asked to define 
integrated STEM, she replied, “I really don’t remember 
what it stands for, haha, maybe the ‘S’ is science?” 
Ironically, her own science experiences were her driver for 
participating in our research. She said, “I mean. . .because 
I don’t have memorable experiences from elementary 
school in science, I kind of want to change that for my 
future students—make it more engaging, and learn and 
incorporate more about STEM.” 

Initial conception of integrated STEM. Jill did not 
have a well-defined initial conception of integrated STEM. 
Furthermore, she suggested that traditional science units 
and integrated STEM units do not differ. However, she did 
advocate integrated STEM as a more effective approach 
because of more subjects being “meaningfully” taught 
together. “Which one is more effective? Hm. ..probably 
integrated STEM...it expands beyond the science 
unit. ..within STEM you can teach the science unit with 
more meaningful purpose” she said. 

Jill also thought “informal” (formative) assessments 
should be utilized in the integrated STEM classroom. She 
stated, “Informally, I would definitely be going around the 
room asking them questions. . .perhaps having a discussion 
with each student to formally assess ‘what did you get out 
of this? Do you see, like, the point we’re making here?’” 
Finally, Jill suggested all STEM disciplines were fully 
connected, “...because now I remember this conversation 
in class, because science, technology, engineering and 
math, you can’t have one without the other.” She started 
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remembering a class discussion from the previous semester 
just as the interview ended. 

Video analysis and reflections. Through the first two 
videos, Jill’s video reflections focused heavily on what 
integrated STEM disciplines were being represented in 
each unit, how often they were represented, and literally 
stating what activities were taking place. When reflecting 
over the control video (roller coaster design), Jill reflected 
under the “Example of STEM Integration” category: 
“Design the optimal coaster that’s the longest that will keep 
the marble on,” and later, “Seeing if their coasters work.” 
She did the same for each category of coding (“Teacher 
Pedagogy,” “Students,” “Example of STEM Integration”; 
see Appendix). She emphasized integrated STEM as the set 
of disciplines in her reflections, writing: 


This video does utilize STEM integration into its 
lesson/unit. I could see the students and teachers 
incorporating all of the aspects of STEM: science, 
technology, engineering, and math. 


When asked to discuss how the unit differs from an 
effective science unit, she indirectly referred to group work 
and active learning, and focused on the teacher’s guided 
questioning. 


I felt that the teacher gave the students a lot of 
opportunities to interact with one another. It wasn’t a 
situation where the teacher lectured the students. . It 
was also effective because the teacher probed students 
for questions and misconception. 


Throughout the next three videos, Jill’s language in her 
reflections changed, more closely correlated with using 
contextual knowledge to reason with presented scenarios; 
the second facet of teacher noticing (van Es & Sherin, 
2002). She started spending more time describing activities 
and thought processes related to what constituted an 
integrated STEM unit. For example, after viewing the 
second video (about heat loss) under the “Student” coding 
category she coded: “Students are figuring out where they 
can better construct their model.” Under the “Teacher” 
category she wrote that the “teacher left instructions vague 
so that students could learn for themselves so that they 
wouldn’t do a good job at construction (at first).” When 
reflecting, Jill pointed out the teacher functioning as a 
facilitator, allowing students to fail—one of the main 
components of an effective integrated STEM unit. 

Reflecting on the third video (bridge design), Jill 
mentioned engineering design. Although mentioned or done 
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in all videos, this was the first time she mentioned it. She 
wrote under the “Student” coding category that students 
“Learn about the steps in the engineering design process.” 
Recalling the heat loss unit, she also (surprisingly) reflected 
that allowing the students to fail could have constituted an 
improvement to the unit. She wrote: 


It would have been cool for the students to have been 
able to build a bridge. . .and fail so that they could learn 
that they need to build a bridge that can support the 
force that is being put on it. 


While watching the fourth video, in which second-grade 
students designed and tested hand-pollinators, Jill coded 
under the “Example of STEM Integration” category: 
“Students work together to identify the engineering steps” 
and “students find ways to solve orchid’s problems.” She 
wrote about the teacher “helping students to identify 
problems and solutions.” 


I noticed that all of the students were engaged in the 
lesson and had the opportunity to read, design, and 
build...I really liked that it provided students the 
opportunity to identify ways in which they could 
improve. 


Finally, Jill watched and reflected on the control video 
again. This time around, examples of coding under the 
“Example of STEM Integration” included: “Engineering- 
design with physics of a roller coaster,” and “Students 
discuss kinetic and potential energy with their marbles.” In 
the “Assessment” category she wrote that “(students) write 
down why it was a failure and one modification they would 
make to their design.” Reflecting, Jill expanded that: “AII 
of the components were present and effective in this STEM 
lesson in the science classroom. I really liked seeing how 
they worked together in order to create a lesson that was 
meaningful to the students and promoted higher-order 
thinking.” However, she still referred here to integrated 
STEM as a set of disciplines with associated classroom 
activities. 


Science was used throughout. Technology was used 
after students created a model of their roller 
coaster. ..Engineering was big, designing their roller 
coasters. I really liked that the teacher called her 
students engineers because I feel that it makes the 
lesson more realistic for the students. Math was used to 
create measurements for different pieces of their roller 
coaster. 
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Lesson Procedures: 





Figure I, An excerpt of Jill’s simple machine lesson plan. 


When reflecting about why the unit was effective, she 
talked about students being eager to participate and 
engaged with the engineering design process, writing: 


The students were super eager and willing to participate 
because they were given particular roles (during 
designing). I also liked that all of the components were 
incorporated. I could see myself doing a lesson like this. 


Postinterview. To remind the reader,  Jill’s 
preconception of integrated STEM was undefined. Now she 
laughed, saying “alright haha, (integrated STEM is defined 
as) science, technology, engineering, and math. And 
incorporating STEM into your classroom in regards to a 
science classroom is incorporating all those components to 
make a lesson.” While suggesting that all STEM disciplines 
comprise a more effective integrated STEM unit, she also 
acknowledged not all need to be present all the time. 
“Well. . .after watching the videos and learning more about 
STEM I found that some teachers incorporate some aspects 
of STEM and some teachers incorporate all of the aspects,” 
she said. Jill also emphasized the importance of hands-on 
activities. “I think the more students can actually see things 
in front of them and manipulate them themselves the more 
they’ Il realize it’s real life.” She then mentions how: 


The cool thing. ..was that they (the students) weren’t 
really using the textbook... you’re using more 
resources than the textbook. It’s like, newer concepts 
we’re trying to introduce, so textbooks aren’t going to 
incorporate all of those aspects. 


Most surprisingly, Jill said she had already adopted 
integrated STEM teaching for herself in her teacher 
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1. The teacher will begin by engaging the students in a conversation about simple machines in 
order to review what they learned at Imagination Station for their field trip. 
2. The teacher will then provide students with the opportunity to create their vehicle. 
3. Students will be allowed to explore the given materials and chose what they think will be 
the best in helping them build their simple machine vehicle. 
4. At this time, the teacher will walk around the classroom and provide scaffolding as needed. 
5. After this, the teacher will ask the students to place their simple machine in front of a fan to 
help them observe how a force acting upon their vehicle helps make it go forward. 
6. The teacher will then have a discussion with the students about their simple machines. 
7. The students will then complete their assessments in two ways: 
a. Through writing in their journal 
b. Completing a provided handout 





placement classroom. She said, “Having students fail wasn’t 
something that I would have thought to do until I watched 
that video, and then my mind was blown! I was like ‘Wow, 
we can let the students fail!’” She smiled, letting on that: 


Actually, after that I created a lesson at one of the 
schools I was at this semester, and we talked about 
simple machines, and so they got to build their own 
toilet paper tube simple machine that’s a car, for a 
wheel-and-axle, and so I just gave them the materials 
and just let them go to create it. But it wasn’t until they 
saw my model that they were able to make 
improvements from theirs. So that was definitely 
something I took moving forward. 


Lesson plan. Jill eagerly volunteered her lesson plan. 
While the lesson plan was organized using Bybee’s 5E 
Instructional Model, Jill highlighted all she had learned from 
watching the videos. While doing so, her idea contained 
different integrated STEM content than was found in any 
videos. Figure 1 shows an excerpt of the lesson plan. 

Change in conception. As shown through excerpts 
from the initial interview and throughout the reflections, 
Jill’s conception of integrated STEM changed through the 
course of the intervention. Comparing the first and last 
video reflection (of the same control video), her emphasis 
shifted from more focused merely on the STEM disciplines 
to emphasizing activities taking place in conjunction with 
each discipline. Although Jill still seemed to widely 
consider integrated STEM education centering on each 
STEM discipline being present in the unit, in the 
postinterview she differentiated that not all disciplines need 
to be present in each unit. Throughout the reflections, she 
also highlighted the facilitative role of the teacher, as well 


as teamwork, designing, redesigning, and _ student 
engagement. 
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Case 2: Anne 

Background. Anne was an Anglo-female senior in the 
Elementary Education program. During her _ initial 
interview, she described growing up in a Midwest, urban 
setting. She cited having a lot of experience with diversity, 
and wanted to teach in urban settings. She also talked about 
having positive science experiences and _ influential 
teachers, but how math was always her favorite because of 
the way her teachers “contextualized the problems within 
science or other disciplines.” She talked about teaching 
math and observing science taught in her teacher placement 
classroom, and had previously heard of STEM integration. 

Initial conception of STEM. Anne had a unique initial 
conception of integrated STEM. While she knew that it 
stood for Science, Technology, Engineering, and Math, 
emphasis was placed on integrating STEM with other non- 
STEM disciplines. She said in her initial interview: 


STEM integration would be utilizing those [STEM] 
fields to teach other [disciplines], or vice versa, so like, 
using Language to teach Science, Technology, 
Engineering, Math. Um, I don’t really know I see it 
much different than any other type of integration. . . 


She was also very skeptical of using integrated STEM, 
because as she described it: “I think that if you use one 
[discipline] to teach the other, you set yourself up to get to 
the risk of not actually teaching one, because that focus has 
to be on something, ya know?” 

She thought integrated STEM should be assessed 
formatively through attempting to investigate the “driving 
creativity of it,” and suggested a facilitative role for the 
teacher.” All classes should be characterized by discovery 
learning,” she said, or “mostly student-driven.” 

Video analysis and_ reflections. Anne’s video 
reflections progressed from more of an emphasis on broad 
integration and literal descriptions of activities, to more 
specifically STEM integration and why activities were 
taking place. For the first two videos, Anne focused mainly 
on activities emphasizing integration. Under the “Example 
of STEM Integration” coding category, Anne explicitly 
included the word “integration” into the majority of her 
coding. She wrote: “measuring lengths and recording in 
table is integrating math,” and “using Newton’s Laws of 
Motion to support design and materials used is integrating 
science.” By the second video, she started labeling the 
integration. She coded “using thermometers and 
iPads = technology,” and “math integration with working 
with measurements.” Like Jill’s reflections, rationale for 
activities was absent and activities were literally described. 
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She defined integrated STEM during her reflections 
mainly as the set of disciplines, as well as broad integration 
(STEM vs. Non-STEM). Reflecting on the control video 
she wrote: 


This video is an example of a lesson/unit utilizing 
STEM integration. Each of the content areas that make 
up STEM was involved in the lesson. The students were 
participating in self-discovery learning and integration 
with language arts. 


She offered that the second video case (heat loss) differed 
from a science unit through its real-world context—a 
central component of effective integrated STEM units 
(Moore et al., 2015), saying: 


This lesson was different than effective science lessons 
I have seen because. . .students are working in a realistic 
context—one that is familiar to everyone (the idea of a 
building losing heat). They were able to see how this 
lesson related to their everyday lives, whereas other 
science lessons fail to make that connection/with 
students. 


Reflecting on the third video (which was about bridge- 
building), Anne started labeling activities with multiple 
disciplines. For example, under the “Examples of STEM 
Integration” category she coded: “Math integration by 
counting, engineering with structure application, science 
with force concept.” In the “Student” category she now 
focused on designing and materials. She wrote: “Design a 
bridge idea and explain how each piece will be used,” 
“offer input as to what different materials could be used,” 
and “design bridge and draw design.” In the ““Assessment” 
category she explained “explaining their designs to check 
for reasoning as to why it was strong.” 

When answering the reflection questions that followed 
her coding, Anne pointed out the purposeful integration 
present in the unit. “I think this is good evidence of proper 
integration as it was seamless. The components of STEM 
each served a meaningful and intentional purpose in the 
lesson. For example, the math integration was not simply 
added to meet the math requirement-it had a purpose in 
helping the students learn,” she wrote. 

For the fourth video (designing hand-pollinators), Anne 
interestingly suggested that it was a more interdisciplinary 
unit. Within the first minute of the video under the 
“Example of STEM Integration” category she coded “The 
story talked about engineering and science; this was more 
interdisciplinary than STEM.” In the “Assessment” 
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category, Anne again emphasized why students did certain 
activities, writing: “Talking with small groups to get an idea 
of their thoughts about a design,” In her reflection questions 
she describes in more detail why she did not consider it an 
integrated STEM unit: 


This lesson missed some fundamental components. I 
felt that this teacher played too much of the role of the 
“answer-giver” rather than facilitator. Math was absent 
from the lesson and you cannot have an effective lesson 
that utilizes STEM integration by leaving out one of the 
key components of STEM. 


Instead, Anne labels it as “an effective science unit” as 
students are still able to see the “bigger picture” and 
connect what they learned to the real world. 

Finally, Anne reflected again on the control video. Her 
coding was very similar to the fourth video. Again, in the 
“Example of STEM Integration” category she started by 
coding “The story talked about engineering and science; 
this was more interdisciplinary than integrated STEM.” 
However, she was able to code for all integrated STEM 
disciplines this time, and thus later considered it an 
integrated STEM unit. Under the “Assessment” category 
she wrote about group discussion, formative assessments, 
students reasoning with materials, and seeing if students 
can answer the “why” questions that are asked by the 
teacher following the design task. “If they [the students] can 
(answer), they really understand the concepts and if not 
then they may only have a contextualized understanding of 
the key ideas” she wrote. Reflecting, Anne demonstrated a 
complete understanding of STEM integration, writing: 


Each of the content areas that make up integrated 
STEM were obvious and purposefully utilized in the 
lesson—seamless integration. The students were 
curious, asking questions and seeking understanding 
and answers using math, scientific concepts, models, 
etc. The design process focused on engineering and 
required the use of technology to aid in this process. 


Here she demonstrated an understanding that 
encompassed both concepts and activities. When asked 
how the unit differed from an effective science unit, Anne 
responded that: 


This lesson was effective because it was student- 
centered. The students were working as a group and 
they were communicating their ideas using appropriate 
vocabulary/terminology. The teacher was the facilitator 
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as seen by her prompting and questioning. The students 
were able to share their thinking and reasoning with her. 


Postinterview. To remind the reader, Anne’s initial 
conception of integrated STEM could be characterized as 
the STEM disciplines, plus broad integration. In the 
postinterview, Anne’s integrated STEM definition changed 
slightly, aligned with her video reflections. She stated, 
“STEM is Science, Technology, Engineering, and Math, to 
do it in that order, haha. So, um, I think the integration of it 
is like, seamlessly having all those pieces together and 
having it make sense in the lesson.” Defining “seamlessly,” 
she said that someone coming into the classroom “should 
not be able to tell really” what disciplines are being taught, 
but that they are “cohesive and together.” 

Anne also cited more fully understanding the value of 
performing hands-on activities compared with rote learning 
activities. She said hands-on activities really help students 
to transfer material they’ve learned, and reflected on her 
own K-12 science experience: 


I always thought that like the science units I loved the 
most, they really were integrated STEM lessons. Ya 
know, I think that they were the most misclassified as 
science units, because they did integrate math, and they 
did integrate things like the engineering piece. Like, we 
got to build things and test them. And we used 
resources and materials that could be the technology 
piece. So I think that the science ones that I guess 
weren’t as...fun, haha...they tended towards more of 
drill and practice (rote learning). 


When asked about the effectiveness of integrated STEM, 
she replied that units are much more effective than 
traditional science units because of the real-world context, 
which “could really help kids want to learn it more, and that 
it does really matter.” As far as assessment, she now 
suggested that through design tasks that rubrics should be 
used to assess students. 

Overall, Anne said that the intervention itself was helpful 
to her in understanding integrated STEM. Skeptical of 
teaching using integrated STEM approaches to begin with, 
however, Anne now talked about newly found barriers. Due 
to her familiarity with urban school settings, Anne 
discussed limited resources or funds for teachers as definite 
barriers to teaching using integrated STEM approaches. 
She also talked about the crippling feeling of having so 
many standards to follow. She says sadly that “I guess if I 
saw. . release on like, the leash on teachers, ya know? It’s 
like, there’s so many strict, like, ‘teach this today, do this 
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today,’ and there’s not really any room for creativity, 
or. ..we’re experts in our field! I know this is an effective 
teaching method.” 

Lesson plans. Anne provided us with numerous lesson 
plans, but none were integrated STEM units because the 
school she was currently teaching at did not have adequate 
resources, and furthermore had strict expectations of 
teachers to “meet standards.” Of the five disclosed lessons, 
three of them were math-centered (adding/subtracting/ 
taking measurements using tools) and two covered topics in 
biology (animal relationships/life cycle of a frog). In her 
lessons, Anne clearly emphasized science skills such as 
interpreting data and using evidence to support claims. 

Change in conception. Anne’s conception of integrated 
STEM changed through the course of the intervention. 
Because she had a more-defined initial conception, as well 
as more teaching experiences, changes were better reflected 
in the way that she viewed and discussed integrated STEM. 
Her initial conception highlighted integration—between 
STEM and Non-STEM disciplines. She also was unsure of 
how integrated STEM would be assessed. Following the 
intervention, Anne’s conception became more focused. She 
talked about “seamless” integration and STEM-specific 
activities such as designing within a real-world context in 
which “students’ curiosity and engagement would drive the 
unit.” Her attitude also changed. Whereas she initially 
showed little interest in teaching using STEM approaches, 
following the intervention she had not only expressed 
interest in using integrated STEM teaching, but had clearly 
identified what barriers are currently holding her (and other 
teachers) back from currently doing so. 


Discussion 

Using van Es and Sherin’s (2002) teacher noticing 
framework in conjunction with current video-based 
intervention best practices (observing, analyzing, 
reflecting—Tripp & Rich, 2012) and videos of integrated 
STEM teaching, results presented here suggest that 
participants began to conceptualize integrated STEM 
differently than before the intervention. Findings also echo 
literature suggesting video-based reflection may support 
teachers to consider and evaluate their views of teaching 
and learning (Abell et al., 1998; Seidel, Stiirmer, Blomberg, 
Kobarg, & Schwindt, 2011; Sherin, 2007; Sherin & Russ, 
2014). Participant conceptions became more aligned with 
what Moore et al. (2015) consider to be the six central 
components of every integrated STEM unit, after immersed 
in only 1.25 hours of video. Interestingly, — this 
understanding was most expressed through preinterviews 
and postinterviews, while both participants maintained a 
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focus on integrated STEM as a set of disciplines throughout 
their analyses and reflections. One reason we hypothesize 
for this difference is that specific integrated STEM activities 
were not explicitly named or discussed in the videos, yet 
were evident for the preservice teachers to notice. 
Therefore, we suggest most instances of preservice 
teachers’ using specific integrated STEM-related terms 
were either: (a) previously held pedagogical terminology 
being recognized in the videos and transferred to, or 
recalled during their reflections, or (b) understanding 
gained through watching the videos. Likewise, teachers 
could have also had difficulties with analyzing and 
reflecting versus discussing what they noticed verbally with 
the researchers (van Es & Sherin, 2008). Concurrent 
teacher placement experiences (e.g., implementing 
integrated STEM) may have influenced results as well. 
Similarities in Conceptual Change Surrounding 
STEM Education 

As mentioned above, both participants seemed to begin 
to recognize integrated STEM through video analysis. This 
included not only integrated STEM content and pedagogy, 
but contextualization and assessment as well. 

Integrated STEM content (as well as discipline) was 
largely the anchor by which participants organized their 
coding. This suggested high importance placed on content 
even before the intervention, or perhaps a familiar entry 
point from which to build their conceptions. Pedagogically, 
integrated STEM was denoted as a student-centered 
approach in which the teacher serves as a facilitator, 
highlighted by groupwork and the usage of the engineering 
design process. Design terms such as “failing,” “redesign,” 
and “engineering design process” began to appear in 
participants’ coding as each video was viewed and reflected 
upon. 

Contextualization became recognized as _ setting 
integrated STEM lessons within “real-world scenarios” in 
which students can “apply their knowledge” through hands- 
on activities. Contrasting some initial participant 
skepticism, integrated STEM became an authentic and 
relevant approach. Surrounding assessment, participants 
recognized formative evaluation and the use of rubrics. 
Participants especially pointed out aspects of design 
planning in which students must test and evaluate materials 
toward building their designs. 

Aside from a recognition of specific integrated STEM- 
related activities, both preservice teachers also seemed to 
gain an awareness and appreciation for purposeful 
integration of STEM disciplines. In the preinterview, one of 
the two participants neither mentioned integration nor 
spoke about how STEM disciplines were connected. Yet, in 
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the postinterviews, both participants discussed “seamless 
integration,” in which an observer of an integrated STEM 
lesson cannot tell what discipline is being taught. This was 
an especially encouraging finding based on previous work 
showing a wide range of conceptions surrounding 
integration of STEM disciplines (Radloff & Guzey, 2016). 
Differences in Conceptual Change Surrounding 
STEM Education 

Yet, some aspects of conceptual change also differed. For 
one, Anne’s postintervention conception was more focused 
on fine details of integrated STEM than Jill’s. We suggest 
this was due in part to Anne having more experience with 
teaching and observing STEM disciplines, or in other 
words, being able to better recognize important aspects of 
teaching scenarios (van Es & Sherin, 2002). She also had a 
fairly defined initial conception. While Jill worked on 
creating a conception of integrated STEM, Anne (already 
possessing a conception) was able to flesh out what made 
integrated STEM approaches feasible to utilize in her 
classroom, as well as what barriers existed. Anne also 
focused more on how it was being assessed, and suggested 
one reason she liked the videos so much was that they 
provided her with resources. This suggests that she had 
formed a strong conception and was grateful to collect more 
examples of integrated STEM, although she was not able to 
use them presently. While nonetheless remarkable, Jill’s 
conception seemed broader and less detailed than Anne’s. 
However, following watching and reflecting over the 
videos, Jill felt comfortable enough creating and 
implementing an integrated STEM lesson. We suggest 
perhaps her shallow initial conception and lack of perceived 
barriers had led to her taken actions. An absence of initial 
feelings toward STEM could make one either more or less 
critical of implementing it themselves. Yet, both 
participants mentioned the videos helping positively change 
their conceptions of integrated STEM. 


Conclusions 

Following video observation, analysis, and reflection, 
Anne and Jill appeared to gain both more informed, 
focused conceptions of integrated STEM education, not 
evident during preinterviews. They noticed aspects of 
integrated STEM teaching ranging from pedagogy to 
student interactions, as well as components of effective 
STEM units (Moore et al., 2015). Although reflecting 
seemed difficult, they were both able to internalize and 
reflect on how they could use integrated STEM in their 
own classrooms. This resonated with findings from 
previous video-based intervention literature (Abell et al., 
1998; Sherin, 2007). 
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As integrated STEM approaches become utilized in more 
classrooms, preservice teacher programs need to be modified 
to offer relevant, reform-based teacher instruction and 
professional development (NAE & NRC, 2014; NGSS Lead 
States, 2013). While not generalizable, our findings suggest 
what observing, analyzing, and reflecting on integrated 
STEM teaching approaches could achieve conceptually 
toward that goal. Although more research is needed, so are 
short and effective interventions that offer simple, low-cost 
steps in helping with effective teacher development. 
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Appendix 

Presurvey 

The presurvey consisted of ten open-ended questions to 
get a baseline conception of STEM education. The 
postsurvey was the same as the presurvey, with the 
exception of one added question (did you overall find 
viewing and reflecting on the videos helpful with your 
own teaching and lesson plan creation?). We have 
included the presurvey below. 


. Do you have STEM teaching experience? If so, how long/what type of experience? 

. How would you define STEM Integration, starting with what the S,T.E, and M, stand for? 

. Why do you define STEM in this way? How did you come to this belief? 

. What do you think are the main components of a STEM unit? 

. Now think of an effective science unit. What do you think are the main components of an 
effective science unit? 


. How do you think those components of a STEM unit differ from those of an effective science 
unit? 


. What teaching style have you or would you use to teach a STEM unit? 
. Does this differ from how an effective science unit is taught? If so, how? 
. Which do you think is more effective in respect to student leaming (or do you think they are 
equally effective)? Why? 
10. How do you assess STEM student work? 
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PROBLEMS 


sec 4 RS ay Ted Eisenberg, Section Editor 


This section of the Journal offers readers an opportunity to exchange interesting mathematical problems and solutions. 
Please send them to Ted Eisenberg, Department of Mathematics, Ben-Gurion University, Beer-Sheva, Israel or fax to: 
972-86-477-648. Questions concerning proposals and/or solutions can be sent e-mail to eisenbt@013.net. Solutions to 
previously stated problems can be seen at http://www.ssma.org/publications. 


Solutions to the problems stated in this issue should be posted before June 15, 2017 


* 5439: Proposed by Kenneth Korbin, New York, NY 


so 


Express the roots of the equation ~—; (aie =20x in closed form. 


“Closed form” means that the roots en be expressed in their decimal equivalents. 


° 5440: Proposed by Roger Izard, Dallas, TX 
The vertices of rectangle ABCD are labeled in clockwise order, and point F lines on line segment AB. Prove that 
AD+AG>DF+FC: 


* 5441: Proposed by Larry G. Meyer, Fremont, OH 
In triangle ABC draw a line through the ex-center corresponding to side c so that it is parallel to side c. Extend the angle 
bisectors of A and B to meet the constructed lines at points A’ and B’, respectively. Find the length of A’B’ if given either 





1. Angles A, B, C and the circumradius R 

2. Sides a, b, c 

3. The semiperimeter s, the inradius r, and the exradius r, 
4. Semiperimeter s and side c.6066 


¢ 5442: Proposed by José Luis Diaz-Barrero, BarcelonaTech, Barcelona, Spain. 
Let Z,, be the n™ Lucas number defined by Lo=2, L;=1 and for n > 2,L,=L,-1+Ln-2. Prove that for all n > 0, 


(Ly t+2Ln+1)° Tp oan 
2 L2 2. Se he 
o n+2 ( n a) n 
Coat Lt Teo 


is the cube of a positive integer and determine its value. 


* 5443: Proposed by D. M. Bainetu-Giurgiu, “Matei Basarab” National College, Bucharest, Romania and Neculai 
Stanciu “George Emil Palade” General School, Buzau, Romania 


3 x 
Compute | ———dx. 
P I sin 2x 


° 5444: Proposed by Ovidiu Fi urdu and Alina Sintamarian, Technical University of Cluj-Napoca, Cluj-Napoca, Romania 
Solve in § the equation {(x+1)?}=2x, where {a} denotes the fractional part of a. 
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* 5445: Proposed by Kenneth Korbin, New York, NY 

Find the sides of a triangle with exradii (3, 4, 5). 

* 5446: Proposed by Arsalan Wares, Valdosta State University, Valdosta, GA 

Polygons ABCD, CEFG, and DGHJ are squares. Moreover, point E is on side DC,X=DGN EF, and Y=BC NJH. If 
GX splits square CEFG in regions whose areas are in the ratio 5:19, what part of square DGHJ is shaded? (Shaded 
region in DGHJ is composed of the areas of triangle YHG and trapezoid EXGC.) 


A D 
J 
XE 
Bei ae G 
H 


¢ 5447: Proposed by luliana Trasca , Scornicesti, Romanai 
Show that if x, y, and z is each a positive real number, then 


x6. +y8 .x3 +76. 3 x P+pP4+34+3x-y-z 
x22. 72 =< 7 
¢ 5448: Proposed by Yubal Barrios and Angel Plaza, University of Las Palmas de Gran Canaria, Spain 


Dida os 
Evaluate: lim ” Se ( Ji 
mee OSijsn 1 J 


it+j=n 





* 5449: Proposed by José Luis Diaz-Barrero, Barcelona Tech, Barcelona, Spain 
Without the use of a computer, find the real roots of the equation 


x° 26x +55x7—39x+ 10=(3x—-2) V3x—-2. 
¢ 5450: Proposed by Ovidiu Furdui, Technical University of Cluj-Napoca, Cluj-Napoca, Romania 
Let k be a positive integer. Calculate 
1 pl kk 
LG lee 
0J0 LY Jam 


where |a| denotes the floor (the integer part) of a. 
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